torchrl.utils

Config

Configuration object used by other modules. Can be saved and imported as a YAML file

class torchrl.utils.config.Config(*args, **kwargs)

Bases: object

Configuration object used for initializing an Agent. It maintains the order from which the attributes have been set.

Parameters:configs (Keyword arguments) – Additional parameters that will be stored. Returns:An object containing all configuration details (with possibly nested Config). Return type:Config object

as_dict()

Returns all object attributes as a nested OrderedDict.

Returns:Nested OrderedDict containing all object attributes. Return type:dict

new_section(name, **configs)

Creates a new Config object and add as an attribute of this instance.

Parameters:

• name (str) – Name of the new section.
• configs (Keyword arguments) – Parameters that will be stored in this section, accepts nested parameters.

Examples

Simple use case:

config.new_section('new_section_name', attr1=value1, attr2=value2, ...)

Nested parameters:

config.new_section('new_section_name', attr1=Config(attr1=value1, attr2=value2))

It’s possible to access the variable like so:

config.new_section_name.attr1

save(file_path)

Saves current configuration to a JSON file. The configuration is stored as a nested dictionary (maintaining the order).

Parameters:file_path (str) – Path to write the file

classmethod from_default(name)

Loads configuration from a default agent.

Parameters:name (str) – Name of the desired config file (‘VanillaPG’, add_more) Returns:A configuration object loaded from a JSON file Return type:Config

static load(file_path)

Loads configuration from a JSON file.

Parameters:file_path (str) – Path of the file to be loaded. Returns:A configuration object loaded from a JSON file Return type:Config

Memories

SimpleMemory

class torchrl.utils.memories.SimpleMemory(*args, initial_keys=None, **kwargs)

Bases: dict

A dict whose keys can be accessed as attributes.

Parameters:initial_keys (list of strings) – Each key will be initialized as an empty list.

DefaultMemory

class torchrl.utils.memories.DefaultMemory(*args, **kwargs)

Bases: collections.defaultdict

A defaultdict whose keys can be accessed as attributes.

Logger

class torchrl.utils.logger.Logger(log_dir=None, *, debug=False, log_freq=1)

Common logger used by all agents, aggregates values and print a nice table.

Parameters:log_dir (str) – Path to write logs file.

add_log(name, value, precision=2)

Register a value to a name, this function can be called multiple times and the values will be averaged when logging.

Parameters:

• name (str) – Name displayed when printing the table.
• value (float) – Value to log.
• precision (int) – Decimal points displayed for the value (Default is 2).

add_tf_only_log(name, value, precision=2)

Register a value to a name, this function can be called multiple times and the values will be averaged when logging. Will not display the logs on the console but just write on the file.

Parameters:

• name (str) – Name displayed when printing the table.
• value (float) – Value to log.
• precision (int) – Decimal points displayed for the value (Default is 2).

add_histogram(name, values)

Register a histogram that can be seen at tensorboard.

Parameters:

• name (str) – Name displayed when printing the table.
• value (torch.Tensor) – Value to log.

log(header=None)

Use the aggregated values to print a table and write to the log file.

Parameters:header (str) – Optional header to include at the top of the table (Default is None).

timeit(i_step, max_steps=-1)

Estimates steps per second by counting how many steps passed between each call of this function.

Parameters:

• i_step (int) – The current time step.
• max_steps (int) – The maximum number of steps of the training loop (Default is -1).

Net Builder

auto_input_shape

torchrl.utils.net_builder.auto_input_shape(obj_config, input_shape)

Create the right input parameter for the type of layer

Parameters:

• obj_config (dict) – A dict containing the function and the parameters for creating the object.
• input_shape (list) – The input dimensions.

get_module_list

torchrl.utils.net_builder.get_module_list(config, input_shape, action_info)

Receives a config object and creates a list of layers.

Parameters:

• config (Config) – The configuration object that should contain the basic network structure.
• input_shape (list) – The input dimensions.
• action_info (dict) – Dict containing information about the environment actions (e.g. shape).

Returns:

A list containing all the instantiated layers.

Return type:

list of layers

nn_from_config

torchrl.utils.net_builder.nn_from_config(config, state_info, action_info, body=None, head=None)

Creates a pytorch model following the instructions of config.

Parameters:

• config (Config) – The configuration object that should contain the basic network structure.
• state_info (dict) – Dict containing information about the environment states (e.g. shape).
• action_info (dict) – Dict containing information about the environment actions (e.g. shape).
• body (Module) – If given use it instead of creating (Default is None).
• head (Module) – If given use it instead of creating (Default is None).

Returns:

A torchrl NN (basically a pytorch NN with extended functionalities).

Return type:

torchrl.SequentialExtended

Utils

get_obj

torchrl.utils.utils.get_obj(config)

Creates an object based on the given config.

Parameters:config (dict) – A dict containing the function and the parameters for creating the object. Returns:The created object. Return type:obj

env_from_config

torchrl.utils.utils.env_from_config(config)

Tries to create an environment from a configuration obj.

Parameters:config (Config) – Configuration file containing the environment function. Returns:env – A torchrl environment. Return type:torchrl.envs Raises:AttributeError – If no env is defined in the config obj.

join_transitions

to_np

torchrl.utils.utils.to_np(value)

explained_var

torchrl.utils.utils.explained_var(target, preds)

Calculates the explained variance between two datasets. Useful for estimating the quality of the value function

Parameters:

• target (np.array) – Target dataset.
• preds (np.array) – Predictions array.

Returns:

The explained variance.

Return type:

float

normalize

torchrl.utils.utils.normalize(array)

Normalize an array by subtracting the mean and diving by the std dev.