Skip to content

Instantly share code, notes, and snippets.

@zcox

zcox/gist:0fc76242702fb803c0c1

Created April 12, 2015 19:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save zcox/0fc76242702fb803c0c1 to your computer and use it in GitHub Desktop.
Save zcox/0fc76242702fb803c0c1 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
2015-04-12 14:51:13,231 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1428864489761_0001_000001
2015-04-12 14:51:13,487 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-04-12 14:51:13,501 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-04-12 14:51:13,600 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2015-04-12 14:51:13,616 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (org.apache.hadoop.yarn.security.AMRMTokenIdentifier@787582d3)
2015-04-12 14:51:13,627 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: The specific max attempts: 2 for application: 1. Attempt num: 1 is last retry: false
2015-04-12 14:51:13,634 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2015-04-12 14:51:13,747 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-04-12 14:51:13,755 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-04-12 14:51:14,318 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2015-04-12 14:51:14,388 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2015-04-12 14:51:14,417 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2015-04-12 14:51:14,419 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2015-04-12 14:51:14,420 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2015-04-12 14:51:14,420 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2015-04-12 14:51:14,421 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2015-04-12 14:51:14,427 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2015-04-12 14:51:14,428 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2015-04-12 14:51:14,429 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2015-04-12 14:51:14,437 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Creating intermediate history logDir: [hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate] + based on conf. Should ideally be created by the JobHistoryServer: yarn.app.mapreduce.am.create-intermediate-jh-base-dir
2015-04-12 14:51:14,447 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Perms after creating 493, Expected: 1023
2015-04-12 14:51:14,447 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Explicitly setting permissions to : 1023, rwxrwxrwt
2015-04-12 14:51:14,458 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Perms after creating 488, Expected: 504
2015-04-12 14:51:14,458 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Explicitly setting permissions to : 504, rwxrwx---
2015-04-12 14:51:14,521 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2015-04-12 14:51:14,773 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-mrappmaster.properties,hadoop-metrics2.properties
2015-04-12 14:51:14,838 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-04-12 14:51:14,838 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2015-04-12 14:51:14,848 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1428864489761_0001 to jobTokenSecretManager
2015-04-12 14:51:14,979 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1428864489761_0001 because: not enabled;
2015-04-12 14:51:14,993 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1428864489761_0001 = 1142. Number of splits = 1
2015-04-12 14:51:14,994 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1428864489761_0001 = 1
2015-04-12 14:51:14,994 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1428864489761_0001Job Transitioned from NEW to INITED
2015-04-12 14:51:14,996 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1428864489761_0001.
2015-04-12 14:51:15,026 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2015-04-12 14:51:15,038 INFO [Socket Reader #1 for port 53727] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 53727
2015-04-12 14:51:15,061 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2015-04-12 14:51:15,062 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-04-12 14:51:15,062 INFO [IPC Server listener on 53727] org.apache.hadoop.ipc.Server: IPC Server listener on 53727: starting
2015-04-12 14:51:15,063 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at hadoop.dev.banno.com/172.17.0.91:53727
2015-04-12 14:51:15,128 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-04-12 14:51:15,134 WARN [main] org.apache.hadoop.http.HttpRequestLog: Jetty request log can only be enabled using Log4j
2015-04-12 14:51:15,145 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2015-04-12 14:51:15,150 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2015-04-12 14:51:15,151 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2015-04-12 14:51:15,155 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2015-04-12 14:51:15,155 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2015-04-12 14:51:15,167 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 33615
2015-04-12 14:51:15,167 INFO [main] org.mortbay.log: jetty-6.1.26
2015-04-12 14:51:15,193 INFO [main] org.mortbay.log: Extract jar:file:/tmp/hadoop-root/nm-local-dir/filecache/184/hadoop-yarn-common-2.4.1.jar!/webapps/mapreduce to /tmp/Jetty_0_0_0_0_33615_mapreduce____.7h49dt/webapp
2015-04-12 14:51:15,471 INFO [main] org.mortbay.log: Started SelectChannelConnector@0.0.0.0:33615
2015-04-12 14:51:15,471 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app /mapreduce started at 33615
2015-04-12 14:51:15,826 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2015-04-12 14:51:15,830 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE job_1428864489761_0001
2015-04-12 14:51:15,832 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2015-04-12 14:51:15,833 INFO [Socket Reader #1 for port 35430] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 35430
2015-04-12 14:51:15,837 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-04-12 14:51:15,837 INFO [IPC Server listener on 35430] org.apache.hadoop.ipc.Server: IPC Server listener on 35430: starting
2015-04-12 14:51:15,863 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2015-04-12 14:51:15,864 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2015-04-12 14:51:15,864 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2015-04-12 14:51:15,917 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-04-12 14:51:15,921 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-04-12 14:51:15,924 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at hadoop.dev.banno.com/172.17.0.91:8030
2015-04-12 14:51:16,035 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: 8192
2015-04-12 14:51:16,035 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: default
2015-04-12 14:51:16,040 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2015-04-12 14:51:16,042 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
2015-04-12 14:51:16,051 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1428864489761_0001Job Transitioned from INITED to SETUP
2015-04-12 14:51:16,053 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2015-04-12 14:51:16,102 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1428864489761_0001Job Transitioned from SETUP to RUNNING
2015-04-12 14:51:16,121 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:16,125 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1428864489761_0001_m_000000 Task Transitioned from NEW to SCHEDULED
2015-04-12 14:51:16,126 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1428864489761_0001_r_000000 Task Transitioned from NEW to SCHEDULED
2015-04-12 14:51:16,127 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-04-12 14:51:16,127 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_r_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-04-12 14:51:16,136 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceReqt:1024
2015-04-12 14:51:16,144 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: reduceResourceReqt:1024
2015-04-12 14:51:16,148 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1428864489761_0001, File: hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/root/.staging/job_1428864489761_0001/job_1428864489761_0001_1.jhist
2015-04-12 14:51:17,039 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2015-04-12 14:51:17,089 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=3 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:6144, vCores:0> knownNMs=1
2015-04-12 14:51:17,090 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=6144
2015-04-12 14:51:17,090 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:18,109 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2015-04-12 14:51:18,110 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1428864489761_0001_01_000002 to attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:18,111 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:18,111 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:18,111 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:18,160 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:18,160 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Job jar is not present. Not adding any jar to the list of resources.
2015-04-12 14:51:18,178 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /tmp/hadoop-yarn/staging/root/.staging/job_1428864489761_0001/job.xml
2015-04-12 14:51:18,780 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #0 tokens and #1 secret keys for NM use for launching container
2015-04-12 14:51:18,780 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 1
2015-04-12 14:51:18,780 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2015-04-12 14:51:19,892 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2015-04-12 14:51:19,912 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=3 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:19,921 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1428864489761_0001_01_000002 taskAttempt attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:19,923 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:19,923 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : hadoop.dev.banno.com:47070
2015-04-12 14:51:20,001 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1428864489761_0001_m_000000_0 : 13562
2015-04-12 14:51:20,003 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1428864489761_0001_m_000000_0] using containerId: [container_1428864489761_0001_01_000002 on NM: [hadoop.dev.banno.com:47070]
2015-04-12 14:51:20,006 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2015-04-12 14:51:20,007 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1428864489761_0001_m_000000
2015-04-12 14:51:20,007 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1428864489761_0001_m_000000 Task Transitioned from SCHEDULED to RUNNING
2015-04-12 14:51:21,672 INFO [Socket Reader #1 for port 35430] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1428864489761_0001 (auth:SIMPLE)
2015-04-12 14:51:21,694 INFO [IPC Server handler 1 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1428864489761_0001_m_000002 asked for a task
2015-04-12 14:51:21,695 INFO [IPC Server handler 1 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1428864489761_0001_m_000002 given task: attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:25,080 INFO [IPC Server handler 1 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1428864489761_0001_m_000000_0 is : 0.0
2015-04-12 14:51:25,093 ERROR [IPC Server handler 2 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1428864489761_0001_m_000000_0 - exited : com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:25,094 INFO [IPC Server handler 2 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_0: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:25,096 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_0: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:25,098 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
2015-04-12 14:51:25,099 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1428864489761_0001_01_000002 taskAttempt attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:25,100 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:25,126 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_0 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
2015-04-12 14:51:25,126 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT
2015-04-12 14:51:25,133 WARN [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://192.168.59.103:9000/druid-working/pageviews/2015-04-12T184908.982Z/_temporary/1/_temporary/attempt_1428864489761_0001_m_000000_0
2015-04-12 14:51:25,135 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_0 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED
2015-04-12 14:51:25,142 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:25,142 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 1 failures on node hadoop.dev.banno.com
2015-04-12 14:51:25,144 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_1 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-04-12 14:51:25,146 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1428864489761_0001_m_000000_1 to list of failed maps
2015-04-12 14:51:25,939 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:25,944 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:25,945 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:25,945 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:26,955 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1428864489761_0001_01_000002
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2015-04-12 14:51:26,956 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_0: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigning container Container: [ContainerId: container_1428864489761_0001_01_000003, NodeId: hadoop.dev.banno.com:47070, NodeHttpAddress: hadoop.dev.banno.com:8042, Resource: <memory:1024, vCores:1>, Priority: 5, Token: Token { kind: ContainerToken, service: 172.17.0.91:47070 }, ] to fast fail map
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned from earlierFailedMaps
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1428864489761_0001_01_000003 to attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:26,956 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:26,956 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:26,957 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_1 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2015-04-12 14:51:26,958 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1428864489761_0001_01_000003 taskAttempt attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:26,958 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:27,014 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1428864489761_0001_m_000000_1 : 13562
2015-04-12 14:51:27,014 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1428864489761_0001_m_000000_1] using containerId: [container_1428864489761_0001_01_000003 on NM: [hadoop.dev.banno.com:47070]
2015-04-12 14:51:27,015 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_1 TaskAttempt Transitioned from ASSIGNED to RUNNING
2015-04-12 14:51:27,015 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1428864489761_0001_m_000000
2015-04-12 14:51:27,960 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:28,686 INFO [Socket Reader #1 for port 35430] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1428864489761_0001 (auth:SIMPLE)
2015-04-12 14:51:28,701 INFO [IPC Server handler 0 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1428864489761_0001_m_000003 asked for a task
2015-04-12 14:51:28,701 INFO [IPC Server handler 0 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1428864489761_0001_m_000003 given task: attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:32,031 INFO [IPC Server handler 1 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1428864489761_0001_m_000000_1 is : 0.0
2015-04-12 14:51:32,042 ERROR [IPC Server handler 2 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1428864489761_0001_m_000000_1 - exited : com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:32,043 INFO [IPC Server handler 2 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_1: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:32,045 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_1: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:32,047 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_1 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
2015-04-12 14:51:32,048 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1428864489761_0001_01_000003 taskAttempt attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:32,049 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:32,054 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_1 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
2015-04-12 14:51:32,054 INFO [CommitterEvent Processor #2] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT
2015-04-12 14:51:32,058 WARN [CommitterEvent Processor #2] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://192.168.59.103:9000/druid-working/pageviews/2015-04-12T184908.982Z/_temporary/1/_temporary/attempt_1428864489761_0001_m_000000_1
2015-04-12 14:51:32,059 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_1 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED
2015-04-12 14:51:32,060 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:32,060 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 2 failures on node hadoop.dev.banno.com
2015-04-12 14:51:32,060 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_2 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-04-12 14:51:32,061 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1428864489761_0001_m_000000_2 to list of failed maps
2015-04-12 14:51:32,976 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:32,980 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:32,980 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:32,980 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:33,986 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1428864489761_0001_01_000003
2015-04-12 14:51:33,986 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2015-04-12 14:51:33,986 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigning container Container: [ContainerId: container_1428864489761_0001_01_000004, NodeId: hadoop.dev.banno.com:47070, NodeHttpAddress: hadoop.dev.banno.com:8042, Resource: <memory:1024, vCores:1>, Priority: 5, Token: Token { kind: ContainerToken, service: 172.17.0.91:47070 }, ] to fast fail map
2015-04-12 14:51:33,986 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_1: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
2015-04-12 14:51:33,986 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned from earlierFailedMaps
2015-04-12 14:51:33,987 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1428864489761_0001_01_000004 to attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:33,987 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:33,987 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:33,987 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:3 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:33,987 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:33,987 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_2 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2015-04-12 14:51:33,988 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1428864489761_0001_01_000004 taskAttempt attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:33,988 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:34,031 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1428864489761_0001_m_000000_2 : 13562
2015-04-12 14:51:34,032 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1428864489761_0001_m_000000_2] using containerId: [container_1428864489761_0001_01_000004 on NM: [hadoop.dev.banno.com:47070]
2015-04-12 14:51:34,032 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_2 TaskAttempt Transitioned from ASSIGNED to RUNNING
2015-04-12 14:51:34,032 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1428864489761_0001_m_000000
2015-04-12 14:51:34,990 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:35,673 INFO [Socket Reader #1 for port 35430] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1428864489761_0001 (auth:SIMPLE)
2015-04-12 14:51:35,688 INFO [IPC Server handler 0 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1428864489761_0001_m_000004 asked for a task
2015-04-12 14:51:35,689 INFO [IPC Server handler 0 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1428864489761_0001_m_000004 given task: attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:39,010 INFO [IPC Server handler 1 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1428864489761_0001_m_000000_2 is : 0.0
2015-04-12 14:51:39,019 ERROR [IPC Server handler 2 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1428864489761_0001_m_000000_2 - exited : com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:39,019 INFO [IPC Server handler 2 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_2: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:39,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_2: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:39,023 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_2 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
2015-04-12 14:51:39,024 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1428864489761_0001_01_000004 taskAttempt attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:39,024 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:39,028 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_2 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
2015-04-12 14:51:39,029 INFO [CommitterEvent Processor #3] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT
2015-04-12 14:51:39,032 WARN [CommitterEvent Processor #3] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://192.168.59.103:9000/druid-working/pageviews/2015-04-12T184908.982Z/_temporary/1/_temporary/attempt_1428864489761_0001_m_000000_2
2015-04-12 14:51:39,033 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_2 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED
2015-04-12 14:51:39,034 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:39,034 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: 3 failures on node hadoop.dev.banno.com
2015-04-12 14:51:39,034 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_3 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-04-12 14:51:39,034 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: Blacklisted host hadoop.dev.banno.com
2015-04-12 14:51:39,035 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added attempt_1428864489761_0001_m_000000_3 to list of failed maps
2015-04-12 14:51:40,005 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:3 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:40,009 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:40,010 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: Update the blacklist for application_1428864489761_0001: blacklistAdditions=1 blacklistRemovals=0
2015-04-12 14:51:40,010 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: Ignore blacklisting set to true. Known: 1, Blacklisted: 1, 100%
2015-04-12 14:51:40,010 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:40,010 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:41,014 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: Update the blacklist for application_1428864489761_0001: blacklistAdditions=0 blacklistRemovals=1
2015-04-12 14:51:41,015 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1428864489761_0001_01_000004
2015-04-12 14:51:41,015 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:41,015 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:41,015 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:3 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:41,015 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_2: Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigning container Container: [ContainerId: container_1428864489761_0001_01_000005, NodeId: hadoop.dev.banno.com:47070, NodeHttpAddress: hadoop.dev.banno.com:8042, Resource: <memory:1024, vCores:1>, Priority: 5, Token: Token { kind: ContainerToken, service: 172.17.0.91:47070 }, ] to fast fail map
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned from earlierFailedMaps
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1428864489761_0001_01_000005 to attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=5120
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1
2015-04-12 14:51:42,020 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:42,020 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved hadoop.dev.banno.com to /default-rack
2015-04-12 14:51:42,021 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_3 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2015-04-12 14:51:42,022 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1428864489761_0001_01_000005 taskAttempt attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:42,022 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:42,071 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1428864489761_0001_m_000000_3 : 13562
2015-04-12 14:51:42,072 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1428864489761_0001_m_000000_3] using containerId: [container_1428864489761_0001_01_000005 on NM: [hadoop.dev.banno.com:47070]
2015-04-12 14:51:42,072 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_3 TaskAttempt Transitioned from ASSIGNED to RUNNING
2015-04-12 14:51:42,072 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1428864489761_0001_m_000000
2015-04-12 14:51:43,024 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1428864489761_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:5120, vCores:-1> knownNMs=1
2015-04-12 14:51:43,734 INFO [Socket Reader #1 for port 35430] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1428864489761_0001 (auth:SIMPLE)
2015-04-12 14:51:43,751 INFO [IPC Server handler 3 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1428864489761_0001_m_000005 asked for a task
2015-04-12 14:51:43,751 INFO [IPC Server handler 3 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1428864489761_0001_m_000005 given task: attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:47,083 INFO [IPC Server handler 0 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1428864489761_0001_m_000000_3 is : 0.0
2015-04-12 14:51:47,092 ERROR [IPC Server handler 3 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1428864489761_0001_m_000000_3 - exited : com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:47,092 INFO [IPC Server handler 3 on 35430] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_3: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:47,094 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1428864489761_0001_m_000000_3: Error: com.metamx.common.RE: Failure on row[{"eventId":"e1", "timestamp":"2015-03-24T14:00:00Z", "userId":"u1", "url":"http://site.com/1"}]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:98)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
at io.druid.indexer.HadoopDruidIndexerConfig.getBucket(HadoopDruidIndexerConfig.java:350)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:231)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:94)
... 9 more
2015-04-12 14:51:47,096 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_3 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
2015-04-12 14:51:47,097 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1428864489761_0001_01_000005 taskAttempt attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:47,098 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:47,100 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_3 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
2015-04-12 14:51:47,101 INFO [CommitterEvent Processor #4] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: TASK_ABORT
2015-04-12 14:51:47,106 WARN [CommitterEvent Processor #4] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete hdfs://192.168.59.103:9000/druid-working/pageviews/2015-04-12T184908.982Z/_temporary/1/_temporary/attempt_1428864489761_0001_m_000000_3
2015-04-12 14:51:47,106 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_m_000000_3 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED
2015-04-12 14:51:47,108 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1428864489761_0001_m_000000 Task Transitioned from RUNNING to FAILED
2015-04-12 14:51:47,108 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2015-04-12 14:51:47,109 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Job failed as tasks failed. failedMaps:1 failedReduces:0
2015-04-12 14:51:47,111 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1428864489761_0001Job Transitioned from RUNNING to FAIL_WAIT
2015-04-12 14:51:47,111 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1428864489761_0001_r_000000 Task Transitioned from SCHEDULED to KILL_WAIT
2015-04-12 14:51:47,111 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1428864489761_0001_r_000000_0 TaskAttempt Transitioned from UNASSIGNED to KILLED
2015-04-12 14:51:47,111 INFO [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Processing the event EventType: CONTAINER_DEALLOCATE
2015-04-12 14:51:47,111 ERROR [Thread-51] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Could not deallocate container for task attemptId attempt_1428864489761_0001_r_000000_0
2015-04-12 14:51:47,111 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1428864489761_0001_r_000000 Task Transitioned from KILL_WAIT to KILLED
2015-04-12 14:51:47,112 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1428864489761_0001Job Transitioned from FAIL_WAIT to FAIL_ABORT
2015-04-12 14:51:47,113 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_ABORT
2015-04-12 14:51:47,123 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1428864489761_0001Job Transitioned from FAIL_ABORT to FAILED
2015-04-12 14:51:47,124 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry
2015-04-12 14:51:47,124 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true
2015-04-12 14:51:47,124 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that shouldUnregistered is: true
2015-04-12 14:51:47,124 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2015-04-12 14:51:47,124 INFO [Thread-64] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
2015-04-12 14:51:47,124 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
2015-04-12 14:51:47,125 INFO [Thread-64] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
2015-04-12 14:51:47,160 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/root/.staging/job_1428864489761_0001/job_1428864489761_0001_1.jhist to hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001-1428864666942-root-pageviews%2Dindex%2Dgenerator%2DOptional.of%28%5B2015%2D03%2D24T-1428864707109-0-0-FAILED-default-1428864676045.jhist_tmp
2015-04-12 14:51:47,190 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001-1428864666942-root-pageviews%2Dindex%2Dgenerator%2DOptional.of%28%5B2015%2D03%2D24T-1428864707109-0-0-FAILED-default-1428864676045.jhist_tmp
2015-04-12 14:51:47,194 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/root/.staging/job_1428864489761_0001/job_1428864489761_0001_1_conf.xml to hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001_conf.xml_tmp
2015-04-12 14:51:47,226 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001_conf.xml_tmp
2015-04-12 14:51:47,233 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001.summary_tmp to hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001.summary
2015-04-12 14:51:47,236 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001_conf.xml_tmp to hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001_conf.xml
2015-04-12 14:51:47,238 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001-1428864666942-root-pageviews%2Dindex%2Dgenerator%2DOptional.of%28%5B2015%2D03%2D24T-1428864707109-0-0-FAILED-default-1428864676045.jhist_tmp to hdfs://hadoop.dev.banno.com:9000/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1428864489761_0001-1428864666942-root-pageviews%2Dindex%2Dgenerator%2DOptional.of%28%5B2015%2D03%2D24T-1428864707109-0-0-FAILED-default-1428864676045.jhist
2015-04-12 14:51:47,238 INFO [Thread-64] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2015-04-12 14:51:47,249 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job diagnostics to Task failed task_1428864489761_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0
2015-04-12 14:51:47,250 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is http://hadoop.dev.banno.com:19888/jobhistory/job/job_1428864489761_0001
2015-04-12 14:51:47,264 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for application to be successfully unregistered.
2015-04-12 14:51:48,268 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:1 RackLocal:0
2015-04-12 14:51:48,269 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://hadoop.dev.banno.com:9000 /tmp/hadoop-yarn/staging/root/.staging/job_1428864489761_0001
2015-04-12 14:51:48,271 INFO [Thread-64] org.apache.hadoop.ipc.Server: Stopping server on 35430
2015-04-12 14:51:48,284 INFO [IPC Server listener on 35430] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 35430
2015-04-12 14:51:48,285 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2015-04-12 14:51:48,284 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment