Improving Efficiency in MapReduce by Optimization Algorithm.

In this Work, task-level scheduling algorithms with respect to resource selection and deadline constraints in heterogeneous Map Reduce environment is consider. However, with the advance of computing technologies and the ever-growth of diverse requirements of end-users, a heterogeneous set of resources that take advantages of different network accelerators, machine architectures, and storage hierarchies allow clouds to be more beneficial to the deployments of the Map- Reduce framework for various applications. The heterogeneity is manifested in the through employment of the optimization algorithms in the resource selection process PSO is selected as the auto optimization strategy for Hybrid resources scheduling algorithm (PH-PSO). There are mainly two types of PSO distinguished by different updating rules for calculating the positions and velocities of particles which are task and resource in the map reduce framework. Hyper parameter selection is a kind of continuous optimization problem in the large scale and iterative applications, and feature selection is a kind of binary optimization problem.