scala - Unable to run Spark with Mesos -

i set spark-0.9.1 run on mesos-0.13.0 using steps mentioned here. mesos ui showing 2 workers registered. want run these commands on spark-shell

> scala> val data = 1 10000 data: > scala.collection.immutable.range.inclusive = range(1, 2, 3, 4, 5, 6, > 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, > 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, > 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, > 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, > 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, > 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, > 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, > 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, > 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, > 164, 165, 166, 167, 168, 169, 170...   > scala> val distdata = sc.parallelize(data) distdata: > org.apache.spark.rdd.rdd[int] = parallelcollectionrdd[0] @ > parallelize @ <console>:14 

now when run collect method, following error occurs.

> scala> distdata.filter(_< 10).collect() 14/06/03 19:54:55 info sparkcontext: starting job: collect @ <console>:17 14/06/03 19:54:55 info dagscheduler: got job 0 (collect @ <console>:17) 8 output partitions (allowlocal=false) 14/06/03 19:54:55 info dagscheduler: final stage: stage 0 (collect @ <console>:17) 14/06/03 19:54:55 info dagscheduler: parents of final stage: list() 14/06/03 19:54:55 info dagscheduler: missing parents: list() 14/06/03 19:54:55 info dagscheduler: submitting stage 0 (filteredrdd[1] @ filter @ <console>:17), has no missing parents 14/06/03 19:54:55 info dagscheduler: submitting 8 missing tasks stage 0 (filteredrdd[1] @ filter @ <console>:17) 14/06/03 19:54:55 info taskschedulerimpl: adding task set 0.0 8 tasks 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:0 tid 0 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:0 1338 bytes in 8 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:1 tid 1 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:1 1338 bytes in 0 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:2 tid 2 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:2 1338 bytes in 0 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:3 tid 3 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:3 1338 bytes in 1 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:4 tid 4 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:4 1338 bytes in 0 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:5 tid 5 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:5 1338 bytes in 0 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:6 tid 6 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:6 1338 bytes in 0 ms 14/06/03 19:54:55 info tasksetmanager: starting task 0.0:7 tid 7 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:55 info tasksetmanager: serialized task 0.0:7 1338 bytes in 0 ms 14/06/03 19:54:56 info tasksetmanager: re-queueing tasks 201406031732-3213994176-5050-6320-10 taskset 0.0 14/06/03 19:54:56 warn tasksetmanager: lost tid 5 (task 0.0:5) 14/06/03 19:54:56 warn tasksetmanager: lost tid 7 (task 0.0:7) 14/06/03 19:54:56 warn tasksetmanager: lost tid 1 (task 0.0:1) 14/06/03 19:54:56 warn tasksetmanager: lost tid 3 (task 0.0:3) 14/06/03 19:54:56 info dagscheduler: executor lost: 201406031732-3213994176-5050-6320-10 (epoch 0) 14/06/03 19:54:56 info blockmanagermasteractor: trying remove executor 201406031732-3213994176-5050-6320-10 blockmanagermaster. 14/06/03 19:54:56 info blockmanagermaster: removed 201406031732-3213994176-5050-6320-10 in removeexecutor 14/06/03 19:54:56 info tasksetmanager: starting task 0.0:3 tid 8 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:56 info tasksetmanager: serialized task 0.0:3 1338 bytes in 0 ms 14/06/03 19:54:56 info dagscheduler: host gained in lost list earlier: host-dsrv04.host  14/06/03 19:54:56 info tasksetmanager: starting task 0.0:1 tid 9 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:56 info tasksetmanager: serialized task 0.0:1 1338 bytes in 0 ms 14/06/03 19:54:56 info tasksetmanager: starting task 0.0:7 tid 10 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:56 info tasksetmanager: serialized task 0.0:7 1338 bytes in 0 ms 14/06/03 19:54:56 info tasksetmanager: starting task 0.0:5 tid 11 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:56 info tasksetmanager: serialized task 0.0:5 1338 bytes in 0 ms 14/06/03 19:54:57 info tasksetmanager: re-queueing tasks 201406031732-3213994176-5050-6320-11 taskset 0.0 14/06/03 19:54:57 warn tasksetmanager: lost tid 8 (task 0.0:3) 14/06/03 19:54:57 warn tasksetmanager: lost tid 2 (task 0.0:2) 14/06/03 19:54:57 warn tasksetmanager: lost tid 4 (task 0.0:4) 14/06/03 19:54:57 warn tasksetmanager: lost tid 10 (task 0.0:7) 14/06/03 19:54:57 warn tasksetmanager: lost tid 6 (task 0.0:6) 14/06/03 19:54:57 warn tasksetmanager: lost tid 0 (task 0.0:0) 14/06/03 19:54:57 info dagscheduler: executor lost: 201406031732-3213994176-5050-6320-11 (epoch 1) 14/06/03 19:54:57 info blockmanagermasteractor: trying remove executor 201406031732-3213994176-5050-6320-11 blockmanagermaster. 14/06/03 19:54:57 info blockmanagermaster: removed 201406031732-3213994176-5050-6320-11 in removeexecutor 14/06/03 19:54:57 info dagscheduler: host gained in lost list earlier: host-dsrv05.host  14/06/03 19:54:57 info tasksetmanager: starting task 0.0:0 tid 12 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:0 1338 bytes in 1 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:6 tid 13 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:6 1338 bytes in 0 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:7 tid 14 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:7 1338 bytes in 1 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:4 tid 15 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:4 1338 bytes in 0 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:2 tid 16 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:2 1338 bytes in 0 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:3 tid 17 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:3 1338 bytes in 1 ms 14/06/03 19:54:57 info tasksetmanager: re-queueing tasks 201406031732-3213994176-5050-6320-11 taskset 0.0 14/06/03 19:54:57 warn tasksetmanager: lost tid 14 (task 0.0:7) 14/06/03 19:54:57 warn tasksetmanager: lost tid 16 (task 0.0:2) 14/06/03 19:54:57 warn tasksetmanager: lost tid 12 (task 0.0:0) 14/06/03 19:54:57 info dagscheduler: executor lost: 201406031732-3213994176-5050-6320-11 (epoch 2) 14/06/03 19:54:57 info blockmanagermasteractor: trying remove executor 201406031732-3213994176-5050-6320-11 blockmanagermaster. 14/06/03 19:54:57 info blockmanagermaster: removed 201406031732-3213994176-5050-6320-11 in removeexecutor 14/06/03 19:54:57 info dagscheduler: host gained in lost list earlier: host-dsrv05.host  14/06/03 19:54:57 info tasksetmanager: starting task 0.0:0 tid 18 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:0 1338 bytes in 0 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:2 tid 19 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:2 1338 bytes in 0 ms 14/06/03 19:54:57 info tasksetmanager: starting task 0.0:7 tid 20 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:57 info tasksetmanager: serialized task 0.0:7 1338 bytes in 0 ms 14/06/03 19:54:58 info tasksetmanager: re-queueing tasks 201406031732-3213994176-5050-6320-10 taskset 0.0 14/06/03 19:54:58 warn tasksetmanager: lost tid 17 (task 0.0:3) 14/06/03 19:54:58 warn tasksetmanager: lost tid 11 (task 0.0:5) 14/06/03 19:54:58 warn tasksetmanager: lost tid 13 (task 0.0:6) 14/06/03 19:54:58 warn tasksetmanager: lost tid 9 (task 0.0:1) 14/06/03 19:54:58 warn tasksetmanager: lost tid 15 (task 0.0:4) 14/06/03 19:54:58 info dagscheduler: executor lost: 201406031732-3213994176-5050-6320-10 (epoch 3) 14/06/03 19:54:58 info blockmanagermasteractor: trying remove executor 201406031732-3213994176-5050-6320-10 blockmanagermaster. 14/06/03 19:54:58 info blockmanagermaster: removed 201406031732-3213994176-5050-6320-10 in removeexecutor 14/06/03 19:54:58 info dagscheduler: host gained in lost list earlier: host-dsrv04.host  14/06/03 19:54:58 info tasksetmanager: starting task 0.0:4 tid 21 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:58 info tasksetmanager: serialized task 0.0:4 1338 bytes in 0 ms 14/06/03 19:54:58 info tasksetmanager: starting task 0.0:1 tid 22 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:58 info tasksetmanager: serialized task 0.0:1 1338 bytes in 0 ms 14/06/03 19:54:58 info tasksetmanager: starting task 0.0:6 tid 23 on executor 201406031732-3213994176-5050-6320-11: host-dsrv05.host  (process_local) 14/06/03 19:54:58 info tasksetmanager: serialized task 0.0:6 1338 bytes in 0 ms 14/06/03 19:54:58 info tasksetmanager: starting task 0.0:5 tid 24 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:58 info tasksetmanager: serialized task 0.0:5 1338 bytes in 1 ms 14/06/03 19:54:58 info tasksetmanager: starting task 0.0:3 tid 25 on executor 201406031732-3213994176-5050-6320-10: host-dsrv04.host  (process_local) 14/06/03 19:54:58 info tasksetmanager: serialized task 0.0:3 1338 bytes in 0 ms 14/06/03 19:54:59 info tasksetmanager: re-queueing tasks 201406031732-3213994176-5050-6320-11 taskset 0.0 14/06/03 19:54:59 warn tasksetmanager: lost tid 23 (task 0.0:6) 14/06/03 19:54:59 warn tasksetmanager: lost tid 20 (task 0.0:7) 14/06/03 19:54:59 error tasksetmanager: task 0.0:7 failed 4 times; aborting job 14/06/03 19:54:59 info dagscheduler: failed run collect @ <console>:17 14/06/03 19:54:59 info dagscheduler: executor lost: 201406031732-3213994176-5050-6320-11 (epoch 4) 14/06/03 19:54:59 info blockmanagermasteractor: trying remove executor 201406031732-3213994176-5050-6320-11 blockmanagermaster. 14/06/03 19:54:59 info blockmanagermaster: removed 201406031732-3213994176-5050-6320-11 in removeexecutor 14/06/03 19:54:59 info dagscheduler: host gained in lost list earlier: host-dsrv05.host  org.apache.spark.sparkexception: job aborted: task 0.0:7 failed 4 times (most recent failure: unknown)         @ org.apache.spark.scheduler.dagscheduler$$anonfun$org$apache$spark$scheduler$dagscheduler$$abortstage$1.apply(dagscheduler.scala:1020)         @ org.apache.spark.scheduler.dagscheduler$$anonfun$org$apache$spark$scheduler$dagscheduler$$abortstage$1.apply(dagscheduler.scala:1018)         @ scala.collection.mutable.resizablearray$class.foreach(resizablearray.scala:59)         @ scala.collection.mutable.arraybuffer.foreach(arraybuffer.scala:47)         @ org.apache.spark.scheduler.dagscheduler.org$apache$spark$scheduler$dagscheduler$$abortstage(dagscheduler.scala:1018)         @ org.apache.spark.scheduler.dagscheduler$$anonfun$processevent$10.apply(dagscheduler.scala:604)         @ org.apache.spark.scheduler.dagscheduler$$anonfun$processevent$10.apply(dagscheduler.scala:604)         @ scala.option.foreach(option.scala:236)         @ org.apache.spark.scheduler.dagscheduler.processevent(dagscheduler.scala:604)         @ org.apache.spark.scheduler.dagscheduler$$anonfun$start$1$$anon$2$$anonfun$receive$1.applyorelse(dagscheduler.scala:190)         @ akka.actor.actorcell.receivemessage(actorcell.scala:498)         @ akka.actor.actorcell.invoke(actorcell.scala:456)         @ akka.dispatch.mailbox.processmailbox(mailbox.scala:237)         @ akka.dispatch.mailbox.run(mailbox.scala:219)         @ akka.dispatch.forkjoinexecutorconfigurator$akkaforkjointask.exec(abstractdispatcher.scala:386)         @ scala.concurrent.forkjoin.forkjointask.doexec(forkjointask.java:260)         @ scala.concurrent.forkjoin.forkjoinpool$workqueue.runtask(forkjoinpool.java:1339)         @ scala.concurrent.forkjoin.forkjoinpool.runworker(forkjoinpool.java:1979)         @ scala.concurrent.forkjoin.forkjoinworkerthread.run(forkjoinworkerthread.java:107) >  >  > scala> 14/06/03 19:55:00 info tasksetmanager: re-queueing tasks > 201406031732-3213994176-5050-6320-10 taskset 0.0 14/06/03 > 19:55:00 info taskschedulerimpl: removed taskset 0.0, tasks have > completed, pool 14/06/03 19:55:00 info dagscheduler: executor > lost: 201406031732-3213994176-5050-6320-10 (epoch 5) 14/06/03 19:55:00 > info blockmanagermasteractor: trying remove executor > 201406031732-3213994176-5050-6320-10 blockmanagermaster. 14/06/03 > 19:55:00 info blockmanagermaster: removed > 201406031732-3213994176-5050-6320-10 in removeexecutor > 14/06/03 19:55:00 info dagscheduler: host gained in lost > list earlier: host-dsrv04.host  

i've checked configuration of spark many times , looks fine me. ideas might have gone wrong?

-- thanks

as turns out tar file wasn't created properly. recreated , working fine now. sorry trouble.