Factorized POMDPs

---

source

Factorized_POMDP

def Factorized_POMDP(
    percept_category_sizes:list[int], # list of sizes of each percept category
    factorized_observation_function:NDArray[numpy.float64], # An NxM array, where N is the number of env states and M = sum(percept_category_sizes)
    transition_function:NDArray[numpy.float64], # An NxNxA array, where A is the number of actions. Rows in each slice contain probability distributions.
    initial_state:int=0, # Start state of POMDP
):

A general construtor for factorized POMDPs.

These differ from standard POMDPs in that the transition function indicates the probability distribution over states in each percept category, rather than over all possible percepts

Example Factorized POMDP

This example factorized POMDP is a two state environment. Each state produces a two-category percept. In state 0, both categories are more likely to be 0. In state 1, both categories are more likely to be 1. The second category has a higher error rate in both states.

#Set up factorized POMDP
percept_category_sizes = [2, 2]  # Two categories, each with 2 states

factorized_observation_function = np.array(
    [[0.9, 0.1, 0.8, 0.2],  # State 0
     [0.1, 0.9, 0.2, 0.8]]  # State 1
    )  
transition_function = np.array(
    [[0.,1.],
     [1.,0.]])[:,:,np.newaxis] #2-state cycle, single action

example_factorized_env = Factorized_POMDP(percept_category_sizes, factorized_observation_function, transition_function)

#Set up simulation
T = 10
observed_percepts = ["None"] * T #data structure for storing observations

#simulate for T steps and store observations   
for t in range(T):
    observed_percepts[t] = example_factorized_env.get_observation()
    example_factorized_env.transition(0) #use action 0
    
observed_percepts

[array([0, 0]),
 array([1, 1]),
 array([0, 0]),
 array([1, 1]),
 array([0, 0]),
 array([1, 1]),
 array([0, 0]),
 array([1, 1]),
 array([0, 1]),
 array([1, 1])]

Prebuilt Factorized POMDPs

Factorized Noisy Cycle

---

source

Factorized_Noisy_Cycle

def Factorized_Noisy_Cycle(
    cycle_length:int, # number of states in the cycle
    percept_category_count:int, # number of percept categories
    category_size:int, # number of states in each percept category
    error_rate:float=0.1, # error rate for each category
    initial_state:int=0
):

A factorized POMPDP with a determinstic cycle between states with uniform error rates for all percept categories

Example Factorized Noisy Cycle

#Set up factorized POMDP
cycle_length = 5
percept_category_count = 2
category_size = 2
error_rate = 0.1
example_factorized_noisy_cycle = Factorized_Noisy_Cycle(cycle_length, percept_category_count, category_size, error_rate)

#Set up Simulation
T = 20 #total time steps to simulate
observed_percepts = ["None"] * T #data structure for storing observations

print(f'Observation Function: \n {example_factorized_noisy_cycle.factorized_observation_function}')
print('Simulation Results:')
#simulate for T steps and store observations
for t in range(T):
    observed_percepts[t] = example_factorized_noisy_cycle.get_observation()
    print(f'state: {example_factorized_noisy_cycle.state}, percept: {observed_percepts[t]}')
    example_factorized_noisy_cycle.transition(0) #use action 0
    #| export
import warnings
import numpy as np
from typing import List, Dict, Any, Optional

Observation Function: 
 [[0.1 0.9 0.9 0.1]
 [0.9 0.1 0.1 0.9]
 [0.9 0.1 0.9 0.1]
 [0.9 0.1 0.1 0.9]
 [0.1 0.9 0.9 0.1]]
Simulation Results:
state: 0, percept: [1 0]
state: 1, percept: [0 1]
state: 2, percept: [0 0]
state: 3, percept: [0 1]
state: 4, percept: [1 1]
state: 0, percept: [1 1]
state: 1, percept: [0 1]
state: 2, percept: [0 0]
state: 3, percept: [1 1]
state: 4, percept: [1 0]
state: 0, percept: [1 0]
state: 1, percept: [1 1]
state: 2, percept: [0 0]
state: 3, percept: [0 1]
state: 4, percept: [1 0]
state: 0, percept: [1 0]
state: 1, percept: [0 1]
state: 2, percept: [0 0]
state: 3, percept: [0 1]
state: 4, percept: [1 1]

---

source

List_Sequencer

def List_Sequencer(
    item_size:int, # number of percept categories per item (K)
    item_category_size:int, # number of valid values per category (C), values are 1..C
    series_length:int, # items per series
    reuse_items:bool=True, # If true, items can be reused in different lists. Items are never reused for the same list
    stimulus_duration:int=1, # time steps for which an item (or probe stimulus) is presented
    mean_timeout_interval:float=4, # μ: desired mean timeout length (Binomial)
    var_time_out_interval:float=2, # σ²: desired variance (should be < μ for Binomial under-dispersion)
    num_series:int=3, # number of series to present
    seed:Optional[int]=None, # random seed for reproducibility (use the same seed to reproduce)
    enable_probe:bool=False, # if True, adds a state to each percept category for a probe cue stimulus, enabling probe recognition
):

A factorized POMDP that produces a specific percept sequence that emulates classic series recognition tasks of human and animal memory

Example List Sequencer

example_list_sequence = List_Sequencer(
    item_size = 2,
    item_category_size= 2,
    series_length= 4,
    num_series = 2
)

example_list_sequence.percept_sequence

array([[2, 1],
       [2, 2],
       [1, 1],
       [1, 2],
       [0, 0],
       [0, 0],
       [0, 0],
       [0, 0],
       [0, 0],
       [2, 2],
       [1, 2],
       [2, 1],
       [1, 1]])

---

source

Series_Recognition_Probe

def Series_Recognition_Probe(
    item_size:int, # number of percept categories per item (K)
    item_category_size:int, # number of valid values per category (C), values are 1..C
    series_length:int, # items per series
    reuse_items:bool=True, stimulus_duration:int=1, # time steps for which an item (or probe stimulus) is presented
    mean_timeout_interval:float=4, # μ: desired mean timeout length (Binomial)
    var_time_out_interval:float=2, # σ²: desired variance (should be < μ for Binomial under-dispersion)
    retention_interval:int=4, # number of retention percept steps after final series
    probe_cue_duration:int=2, # number of time steps for unique cue prior to probe
    num_series:int=3, # number of series to present
    probe_position:int=-1, # -1 novel; 0 sample from earlier (not last) series; 1..L index into last series
    seed:Optional[int]=None, # random seed for reproducibility (use the same seed to reproduce)
):

A factorized POMDP that produces a specific percept sequence that emulates classic series recognition tasks of human and animal memory

Example Series Recognition Probe

test_series_recognition = Series_Recognition_Probe(
    item_size= 4, 
    item_category_size=3, 
    mean_timeout_interval=2,
    var_time_out_interval=0.5,
    series_length=4, 
    retention_interval=2, 
    num_series=2, 
    probe_position= 3)
test_series_recognition.percept_sequence

array([[2, 3, 3, 2],
       [3, 2, 3, 1],
       [3, 3, 2, 3],
       [1, 1, 3, 2],
       [0, 0, 0, 0],
       [2, 1, 2, 3],
       [2, 2, 3, 2],
       [1, 3, 3, 1],
       [2, 3, 2, 1],
       [0, 0, 0, 0],
       [0, 0, 0, 0],
       [4, 4, 4, 4],
       [4, 4, 4, 4],
       [1, 3, 3, 1]], dtype=int64)