Title

Saturday, 7 February 2015

What is the principle ? When Spark deal with data bigger than memory capacity?


As I know , Spark use memory to cache the data and then compute the data in memory.But what if the data is bigger than memory? I could read the source code ,but I don't know which class do the schedule job? Or could you explain the principle of how Spark deal with this question?

If this question is vague ,I will explain it later . Sorry ,I 'm not a native speaker of English..and sorry I'm new to Spark if you think this question is too simple.

Answer

No comments:

Post a Comment