-
Notifications
You must be signed in to change notification settings - Fork 114
Introduction
This page introduces the features of Fenzo, which will be discussed in greater detail on other pages.
Contents:
Fenzo implements a task scheduler for tasks that run in a cluster. It lets Mesos handle the networking infrastructure, fault tolerance primitives, and resource isolation for launching these tasks; Fenzo concentrates on optimizing the scheduling of tasks. You use Fenzo by incorporating it into the framework (or frameworks) that you develop on top of Mesos.
The additional fenzo-triggers utility library allows you to integrate cron-like triggers into Fenzo. It is not documented in this wiki.
Often, a task may be appropriate for more than one host. You want to choose the best host for the task, being aware that by choosing a host for one task you also affect future task assignments.
One strategy for choosing a host for a task is the first-fit strategy (choosing the first host you find that is fit for the task). This is computationally fast, but it may be sub-optimal. For example, it may fragment the resources on the host and make it less able to fit a larger task in the future, or it may place tasks with widely varying completion times on the same host, which would prevent the host from being considered for removal from the cluster if needed (such removal can be beneficial for scaling down an elastic cluster when demand tapers, or to reduce power consumption in the data center).
On the other hand, it can be computationally expensive to find optimal assignments for all tasks on a large number of hosts, even if you do not reassign running tasks.
Fenzo adopts a strategy of scoring hosts for a task placement, with higher scores indicating better “fitness.” This scoring is performed by a fitness calculator plugin that rates the fitness of a task on a given host. Fenzo includes some built-in plugins that you can use for some common bin packing use cases. You can also write your own custom plugins if none of the built-in plugins meet your needs.
To govern the trade-off between assignment optimization and speed, you provide a function to Fenzo that determines if the fitness of a host is “good enough.” If that function says that the fitness of a host is good enough, Fenzo will stop its search for a more optimal host. Fitness evaluators return a value between 0.0 and 1.0, with 0.0 indicating that the task doesn’t fit, and 1.0 indicating that the task fits perfectly. Values between the extremes indicate the relative degree of fitness.
In this way, Fenzo can evaluate fitness on a subset of the hosts, until one is found with a good enough fitness value. You can dynamically control the speed of task assignment and its trade off with fitness optimization as needed by changing this “good enough” function.
You may also apply certain constraints to tasks, such that a host must meet these constraints before Fenzo will consider assigning the task to that host. Constraints may be “hard” or “soft.” Fenzo evaluates the hard constraints first. If a host fails to meet a hard constraint, Fenzo will not consider that host for the task. If that host does meet all hard constraints, Fenzo will then evaluate the soft constraints along with the fitness calculators: The fitness score for a host is an aggregation of the results of the fitness calculators and the soft constraints.
You have a variety of options when you first initialize, or build, your Fenzo scheduler.
- Resource Allocation Limits: You can limit the amount of a variety of system resources that will be available to the tasks in specified task groups.
- Fitness Evaluation: You can tell the Fenzo scheduler how it should evaluate hosts to determine how well suited they are for tasks.
- Autoscaling: You can tell the Fenzo scheduler the circumstances under which it should perform autoscaling and what criteria it should use to make autoscaling decisions.
- Offer Expiry: You can tell the Fenzo scheduler how long to hold on to an unused resource offer before discarding it.
The Fenzo scheduler selects potential hosts to assign tasks to based on whether the hosts have sufficient resources to successfully execute the tasks. Each task has its own resource requirements, each host offer lists its available resources, and Fenzo maintains an accounting of which resources are available and which are already assigned on each host.
But frequently there are many hosts that meet these baseline qualifications. You can fine-tune how the Fenzo scheduler assigns tasks to compatible hosts by using fitness calculators and constraints. It is also possible to combine multiple fitness calculators and constraints into a single, weighted, task scheduling optimization strategy.
For more information on scheduler options, see Building Your Scheduler.
You can restrict the amount of resources (CPU cores, disk space, memory, and bandwidth) available to a group of tasks, such that a new task in that group will not be assigned even to an available and fit host if that task would cause the group of tasks to exceed those resource limits.
For more information, see Resource Allocation Limits.
You tell the Fenzo scheduler how best to match hosts to tasks by using the two varieties of scheduling optimization plugins: Fitness Calculators and Constraints. You can write your own, or use those that are part of Fenzo.
A “bin packing” fitness calculator attempts to maximize the resource use on a host before distributing tasks to another host. This reduces the number of hosts required and makes it easier to scale down clusters. There are several bin packing fitness calculator plugins that are already built-in to Fenzo:
- CPU Bin Packer: tries to use as many of the CPUs on one host before assigning tasks to a new host
- Memory Bin Packer: tries to use as much of the memory available on one host before assigning tasks to a new host
- Network Bin Packer: tries to use as much of the bandwidth available to one host before assigning tasks to a new host
- CPU/Memory Bin Packer: combines the Memory and CPU strategies listed above
- CPU/Memory/Network Bin Packer: combines the Memory, CPU, and Network strategies listed above
Another way to match hosts to tasks is by means of host constraints. For example, your task may demand that it is the only task to be running on the host, or that it run on a host of a particular type, or that a certain set of tasks each runs in a different zone or on a different host.
You can decide whether to treat any of these constraints as a hard constraint (a simple boolean thumbs-up or thumbs-down) or as a soft constraint (a more nuanced measure of acceptability that behaves like a fitness calculator).
The constraint options that are built-in to Fenzo are the following:
- Balanced Host Attribute Constraint: schedules a set of tasks so that they are distributed evenly among types of hosts, where the type is determined by the values of particular host attributes
- Exclusive Host Constraint: ensures that when a task is assigned to a host, it will be the only task assigned to that host
- Host Attribute Value Constraint: matches a task with a host that has a particular value for a particular host attribute
- Unique Host Attribute Constraint: schedules a set of tasks so that each task ends up on a host that has a different value for some host attribute that you specify
You may also implement your own constraints if none of these suit your needs.
Fenzo regulates Autoscaling in two ways:
- it applies Fenzo’s resource shortfall evaluator to estimate resource needs based on the current task workload
- it applies a set of rules, specific to each autoscaling group, that specify the minimum and maximum number of idle hosts to allow and the cooldown period between autoscaling actions in which other such actions are forbidden
Fenzo includes a variety of methods with which you can get insight into how tasks are being assigned, what resources they are using, what hosts are available and what resources they have, how long the scheduling process is taking, what is causing some tasks to fail to be scheduled, and so forth. For more information, see Insights.