Slow task scheduling algorithm based on node identification
In order to reduce the influence of the slow task, produced in big data processing, a scheduling algorithm (TQST) combining recognition, speculation and seduction of slow task was proposed. First of all, through the judgment of node ability and task execution time, slow node queue, very slow node qu...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Journal on Communications
2014-07-01
|
| Series: | Tongxin xuebao |
| Subjects: | |
| Online Access: | http://www.joconline.com.cn/zh/article/doi/10.3969/j.issn.1000-436x.2014.07.015/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841539775833571328 |
|---|---|
| author | Yun-fei CUI Xin-ming LI Yi LI Dong LIU |
| author_facet | Yun-fei CUI Xin-ming LI Yi LI Dong LIU |
| author_sort | Yun-fei CUI |
| collection | DOAJ |
| description | In order to reduce the influence of the slow task, produced in big data processing, a scheduling algorithm (TQST) combining recognition, speculation and seduction of slow task was proposed. First of all, through the judgment of node ability and task execution time, slow node queue, very slow node queue and slow task queue were established. Secondly, according to the anticipation speculative execution value to decide how to start speculative task. Then, in the basis of node identification, avoid distributing tasks to very slow node, radically reduce slow task production, improve job execution efficiency. The experimental results show that TQST algorithm previous existing slow task scheduling al-gorithm in term of the job response time. |
| format | Article |
| id | doaj-art-ec28f134153a41e6883e70a0d13bfd4f |
| institution | Kabale University |
| issn | 1000-436X |
| language | zho |
| publishDate | 2014-07-01 |
| publisher | Editorial Department of Journal on Communications |
| record_format | Article |
| series | Tongxin xuebao |
| spelling | doaj-art-ec28f134153a41e6883e70a0d13bfd4f2025-01-14T06:43:47ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2014-07-013512212859682867Slow task scheduling algorithm based on node identificationYun-fei CUIXin-ming LIYi LIDong LIUIn order to reduce the influence of the slow task, produced in big data processing, a scheduling algorithm (TQST) combining recognition, speculation and seduction of slow task was proposed. First of all, through the judgment of node ability and task execution time, slow node queue, very slow node queue and slow task queue were established. Secondly, according to the anticipation speculative execution value to decide how to start speculative task. Then, in the basis of node identification, avoid distributing tasks to very slow node, radically reduce slow task production, improve job execution efficiency. The experimental results show that TQST algorithm previous existing slow task scheduling al-gorithm in term of the job response time.http://www.joconline.com.cn/zh/article/doi/10.3969/j.issn.1000-436x.2014.07.015/big dataslow taskspeculative taskMap-Reduce |
| spellingShingle | Yun-fei CUI Xin-ming LI Yi LI Dong LIU Slow task scheduling algorithm based on node identification Tongxin xuebao big data slow task speculative task Map-Reduce |
| title | Slow task scheduling algorithm based on node identification |
| title_full | Slow task scheduling algorithm based on node identification |
| title_fullStr | Slow task scheduling algorithm based on node identification |
| title_full_unstemmed | Slow task scheduling algorithm based on node identification |
| title_short | Slow task scheduling algorithm based on node identification |
| title_sort | slow task scheduling algorithm based on node identification |
| topic | big data slow task speculative task Map-Reduce |
| url | http://www.joconline.com.cn/zh/article/doi/10.3969/j.issn.1000-436x.2014.07.015/ |
| work_keys_str_mv | AT yunfeicui slowtaskschedulingalgorithmbasedonnodeidentification AT xinmingli slowtaskschedulingalgorithmbasedonnodeidentification AT yili slowtaskschedulingalgorithmbasedonnodeidentification AT dongliu slowtaskschedulingalgorithmbasedonnodeidentification |