a field, a species) depending on the current state of all the considered entities. A global policy is a set of decision rules that prescribe the actions to apply in any particular entity (e.g.

In general, such methods do not compute an optimal global policy for a given objective, but only an approximate one. Several approaches have been proposed for solving MDPs with multidimensional state and action spaces (FA-FMDPs ). However, their direct application to domains like ecology or agroecology is difficult when there are a large number of state variables together with a large number of action variables. There exist several freely available toolboxes for solving Markov Decision Processes. Solving a MDP amounts to finding the policy that optimizes the expected sum of future rewards, over a given time horizon. A MDP is defined in terms of state variables, action variables, transition probability functions and reward functions. Markov Decision Processes (MDPs ) form a suitable framework for modeling and solving problems of sequential decision under uncertainty. Instead, sequences of decisions must be taken without a precise and deterministic knowledge of the potentially delayed effects of the decisions on the system. Īnother feature of these management problems is that the objective is a long-term one: biodiversity and production must be preserved in a sustainable way and decisions are not taken once and for all. Lastly, management of long-term pesticide durability must be applied at the landscape level for fungicides, insecticides, and herbicides. A given proportion of refuge fields must be maintained at the landscape level in order to limit adaptation of insects to genetically modified Bt crops. In addition, spatial dispersion of pests and beneficials create spatial dependencies between fields and other habitats. For instance, erosion problems must be managed collectively at the catchment basin level. In agroecology, many processes occur at levels higher than the field level because interactions take place among landscape components (commercial fields and interstitial spaces) through biotic and abiotic flows. Ecosystem services are usually expected at the regional level, while management actions are applied at the field level. The choice of the target species/habitats depends strongly on the ecological interaction network between the species and habitats. For instance, for optimizing biodiversity conservation, protection actions may target only a few species or habitats, while the whole biodiversity is of interest. In addition, management actions are applied at a local level while the objective is often defined at a larger level. Management problems in ecology and agroecology are complex because several components must be managed together while spatial interactions occur among them. This example shows how GMDPtoolbox can be used as a tool to support expert thinking.

We provide an illustration of the use of the toolbox on a realistic crop disease management problem: the design of long term management policy of blackleg of canola using an optimal combination of three possible cultural levers. We present GMDPtoolbox, a Matlab toolbox which can be used both for the design of new management policies and for comparing policies by simulation. However, up to now, no “easy-to-use” implementation of this framework was available. This framework has already been successfully applied to forest management and invasive species management. The provided policies are adaptive, meaning that management actions are decided at each time step (for instance yearly) and the chosen actions depend on the current system state. Spatial interactions are easily modeled and integrated control policies (combining several action levers) can be designed through optimization. The framework of Graph-Based Markov Decision Processes (GMDP) is well adapted to the qualitative modeling of such problems of sequential decision under uncertainty. Furthermore, the objectives pursued when managing ecological systems or agroecosystems are usually long term objectives, such as biodiversity conservation or sustainable crop production. Decision choices must be made under uncertainty on the results of the actions and on the system dynamics. Several components must be managed together while they strongly interact spatially. Designing management policies in ecology and agroecology is complex.

Matlab optimization toolbox ubc