cike0415 发表于 2016-12-11 09:23:35

hadoop&spark mapreduce对比 & 框架设计和理解

Hadoop MapReduce:

MapReduce在每次执行的时候都要从磁盘读数据,计算完毕后都要把数据放到磁盘

spark map reduce:



 
 

RDD is everything for dev:

Basic Concepts:


Graph RDD:

Spark Runtime:

schedule:

Depency Type:

Scheduler Optimizations:

Event Flow:

Submit Job:

New Job Instance:

Job In Detail:

executor.launchTask:


Standalone:

 
 
Work Flow:


Standalone detail:


Driver application to Clustor:


Worker Exception:


Executor Exception:


Master Exception:

Master HA:


页: [1]
查看完整版本: hadoop&spark mapreduce对比 & 框架设计和理解