Foraging

This notebook gathers different kinds of environments for foraging and target search in various scenarios, adapted for their use in the reinforcement learning paradigm.

Libs

Helpers

isBetween

Pareto sampling

Random sampling from array with probs

TargetEnv

source

TargetEnv

def TargetEnv(
    Nt:int=10, # Number of targets.
    L:float=1.3, # Size of the (squared) world.
    r:float=1.5, # Radius with center the target position. It defines the area in which agent detects the target.
    lc:list=[[1], [1]], # Cutoff length. Displacement away from target
    agent_step:int=1, # Displacement of one step. The default is 1.
    num_agents:int=1, # Number of agents that forage at the same time. The default is 1. > 1 not fully implemented
    destructive:bool=False, # True if targets are destructive. The default is False.
    lc_distribution:str='constant', # Distribution from where to sample l_c. Can be 'power_law', 'pareto' or something else. See comments self.check_encounter for explanations
):

Class defining the a Foraging environment with multiple targets and two actions: continue in the same direction and turn by a random angle.

ResetEnv

1D

source

ResetEnv_1D

def ResetEnv_1D(
    L:float=1.3, # Distance to cross to get reward
    D:float=1.0, # Diffusion coefficient
):

Class defining a 1D environment where the agent can either continue walking or reset to the origin. Reward = 1 if the agent crosses the distance L, else = 0.

Search loop with fixed policy an arbitrary environment

source

reset_search_loop

def reset_search_loop(
    T, # Number of steps
    reset_policy, # Reset policy
    env, # Environment
):

Loop that runs the reset environment with a given reset policy.

Parallel search loops for Reset 1D

source

parallel_Reset1D_exp

def parallel_Reset1D_exp(
    T, rates, L, D
):

Runs the Reset 1D loop in parallel for different exponential resetting rates.

source

parallel_Reset1D_sharp

def parallel_Reset1D_sharp(
    T:int, # Number of steps
    resets:<built-in function array>, # Array of resetting times
    L:float, # Distance to cross to get reward
    D:float, # Diffusion coefficient
)-><built-in function array>: # Rewards obtained for each resetting time

Runs the Reset 1D loop in parallel for different sharp resetting times.

2D

source

ResetEnv_2D

def ResetEnv_2D(
    dist_target:float=0.2, # Distance from init position and target
    radius_target:float=0.5, # Radius of the target
    D:float=1.0, # Diffusion coefficient of the walker
):

Class defining a 2D environment where the agent can either continue walking or reset to the origin. Reward = 1 if the agent enters the area defined by the distance and radius of the target.

Parallel search loops for Reset 2D

source

parallel_Reset2D_policies

def parallel_Reset2D_policies(
    T, # Number of steps
    reset_policies, # Array of reset policies
    dist_target, # Distance of the target
    radius_target, # Radius of the target
    D, # Diffusion coefficient of the walker
)-><built-in function array>: # Rewards obtained for each reset policy

Runs the Reset 2D loop in parallel for different sharp resetting times.

source

parallel_Reset2D_exp

def parallel_Reset2D_exp(
    T, # Number of steps
    rates, # Reset rates
    dist_target, # Distance of the target
    radius_target, # Radius of the target
    D, # Diffusion coefficient of the walker
)-><built-in function array>: # Rewards obtained for each exponential rate

Runs the Reset 2D loop in parallel for different sharp resetting times.

source

parallel_Reset2D_sharp

def parallel_Reset2D_sharp(
    T, # Number of steps
    resets, # Array of resetting times
    dist_target, # Distance to target
    radius_target, # Radius of the target
    D, # Diffusion coefficient of the walker
)-><built-in function array>: # Rewards obtained for each sharp resetting time

Runs the Reset 2D loop in parallel for different sharp resetting times.

Turn + Reset Environment

Only 2D is considered here as the 1D case is trivial (the agent would never turn, only reset)

source

TurnResetEnv_2D

def TurnResetEnv_2D(
    dist_target:float=0.2, # Distance from init position and target
    radius_target:float=0.5, # Radius of the target
    agent_step:float=1.0, # Diffusion coefficient of the walker
):

Class defining a Foraging environment with a single target and three possible actions:

  • Continue in the same direction
  • Turn by a random angle
  • Reset to the origin

The agent makes steps of constant length given by agent_step.

Search loop with fixed policy

source

search_loop_turn_reset_sharp

def search_loop_turn_reset_sharp(
    T, # Number of steps
    reset, # Reset time
    turn, # Turn time
    env, # Environment
):

Runs a search loop of T steps. There is a single counter that works as follows:

  • Starts at 0
  • For each turn or continue action gets +1
  • If reset or reach the target is set to 0

nbdev