You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In MR, Spark, we are assuming each mapper or reducer handles portion of data. The data size for each map or reduce instance is at most equal to hdfs block size. So the input size can be estimated by the multipication of of blocksize*number of instances. Is the same assumption hold with Alibaba trace ?
Thank you for releasing these traces. The DAG information is so valuable. It would be great if it could include, the inputs to each job and the input sizes.
The text was updated successfully, but these errors were encountered:
Hello,
In MR, Spark, we are assuming each mapper or reducer handles portion of data. The data size for each map or reduce instance is at most equal to hdfs block size. So the input size can be estimated by the multipication of of blocksize*number of instances. Is the same assumption hold with Alibaba trace ?
Thank you for releasing these traces. The DAG information is so valuable. It would be great if it could include, the inputs to each job and the input sizes.
The text was updated successfully, but these errors were encountered: