flow.core package

Submodules

flow.core.experiment module

flow.core.params module

Objects that define the various meta-parameters of an experiment.

class flow.core.params.AimsunParams(sim_step=0.1, render=False, restart_instance=False, emission_path=None, save_render=False, sight_radius=25, show_radius=False, pxpm=2, network_name='Dynamic Scenario 866', experiment_name='Micro SRC Experiment 867', replication_name='Replication 870', centroid_config_name=None, subnetwork_name=None)[source]

Bases: flow.core.params.SimParams

Aimsun-specific simulation parameters.

Extends SimParams.

Attributes

sim_step : float optional

seconds per simulation step; 0.1 by default

render : str or bool, optional

specifies whether to visualize the rollout(s)

  • False: no rendering

  • True: delegate rendering to sumo-gui for back-compatibility

  • “gray”: static grayscale rendering, which is good for training

  • “dgray”: dynamic grayscale rendering

  • “rgb”: static RGB rendering

  • “drgb”: dynamic RGB rendering, which is good for visualization

restart_instance : bool, optional

specifies whether to restart a simulation upon reset. Restarting the instance helps avoid slowdowns cause by excessive inflows over large experiment runtimes, but also require the gui to be started after every reset if “render” is set to True.

emission_path : str, optional

Path to the folder in which to create the emissions output. Emissions output is not generated if this value is not specified

save_render : bool, optional

specifies whether to save rendering data to disk

sight_radius : int, optional

sets the radius of observation for RL vehicles (meter)

show_radius : bool, optional

specifies whether to render the radius of RL observation

pxpm : int, optional

specifies rendering resolution (pixel / meter)

network_name : str, optional

name of the network generated in Aimsun.

experiment_name : str, optional

name of the experiment generated in Aimsun

replication_name : str, optional

name of the replication generated in Aimsun. When loading an Aimsun template, this parameter must be set to the name of the replication to be run by the simulation; in this case, the network_name and experiment_name parameters are not necessary as they will be obtained from the replication name.

centroid_config_name : str, optional

name of the centroid configuration to load in Aimsun. This parameter is only used when loading an Aimsun template, not when generating one.

subnetwork_name : str, optional

name of the subnetwork to load in Aimsun. This parameter is not used when generating a network; it can be used when loading an Aimsun template containing a subnetwork in order to only load the objects contained in this subnetwork. If set to None or if the specified subnetwork does not exist, the whole network will be loaded.

class flow.core.params.EnvParams(additional_params=None, horizon=inf, warmup_steps=0, sims_per_step=1, evaluate=False, clip_actions=True)[source]

Bases: object

Environment and experiment-specific parameters.

This includes specifying the bounds of the action space and relevant coefficients to the reward function, as well as specifying how the positions of vehicles are modified in between rollouts.

Attributes

additional_params : dict, optional

Specify additional environment params for a specific environment configuration

horizon : int, optional

number of steps per rollouts

warmup_steps : int, optional

number of steps performed before the initialization of training during a rollout. These warmup steps are not added as steps into training, and the actions of rl agents during these steps are dictated by sumo. Defaults to zero

sims_per_step : int, optional

number of sumo simulation steps performed in any given rollout step. RL agents perform the same action for the duration of these simulation steps.

evaluate : bool, optional

flag indicating that the evaluation reward should be used so the evaluation reward should be used rather than the normal reward

clip_actions : bool, optional

specifies whether to clip actions from the policy by their range when they are inputted to the reward function. Note that the actions are still clipped before they are provided to apply_rl_actions.

get_additional_param(key)[source]

Return a variable from additional_params.

class flow.core.params.InFlows[source]

Bases: object

Used to add inflows to a network.

Inflows can be specified for any edge that has a specified route or routes.

add(edge, veh_type, vehs_per_hour=None, probability=None, period=None, depart_lane='first', depart_speed=0, name='flow', begin=1, end=86400, number=None, **kwargs)[source]

Specify a new inflow for a given type of vehicles and edge.

Parameters

edge : str

starting edge for the vehicles in this inflow

veh_type : str

type of the vehicles entering the edge. Must match one of the types set in the Vehicles class

vehs_per_hour : float, optional

number of vehicles per hour, equally spaced (in vehicles/hour). Cannot be specified together with probability or period

probability : float, optional

probability for emitting a vehicle each second (between 0 and 1). Cannot be specified together with vehs_per_hour or period

period : float, optional

insert equally spaced vehicles at that period (in seconds). Cannot be specified together with vehs_per_hour or probability

depart_lane : int or str

the lane on which the vehicle shall be inserted. Can be either one of:

  • int >= 0: index of the lane (starting with rightmost = 0)

  • “random”: a random lane is chosen, but the vehicle insertion is not retried if it could not be inserted

  • “free”: the most free (least occupied) lane is chosen

  • “best”: the “free” lane (see above) among those who allow the vehicle the longest ride without the need to change lane

  • “first”: the rightmost lane the vehicle may use

Defaults to “first”.

depart_speed : float or str

the speed with which the vehicle shall enter the network (in m/s) can be either one of:

  • float >= 0: the vehicle is tried to be inserted using the given speed; if that speed is unsafe, departure is delayed

  • “random”: vehicles enter the edge with a random speed between 0 and the speed limit on the edge; the entering speed may be adapted to ensure a safe distance to the leading vehicle is kept

  • “speedLimit”: vehicles enter the edge with the maximum speed that is allowed on this edge; if that speed is unsafe, departure is delayed

Defaults to 0.

name : str, optional

prefix for the id of the vehicles entering via this inflow. Defaults to “flow”

begin : float, optional

first vehicle departure time (in seconds, minimum 1 second). Defaults to 1 second

end : float, optional

end of departure interval (in seconds). This parameter is not taken into account if ‘number’ is specified. Defaults to 24 hours

number : int, optional

total number of vehicles the inflow should create (due to rounding up, this parameter may not be exactly enforced and shouldn’t be set too small). Default: infinite (c.f. ‘end’ parameter)

kwargs : dict, optional

see Note

get()[source]

Return the inflows of each edge.

class flow.core.params.InitialConfig(shuffle=False, spacing='uniform', min_gap=0, perturbation=0.0, x0=0, bunching=0, lanes_distribution=inf, edges_distribution='all', additional_params=None)[source]

Bases: object

Initial configuration parameters.

These parameters that affect the positioning of vehicle in the network at the start of a rollout. By default, vehicles are uniformly distributed in the network.

Attributes

shuffle : bool, optional # TODO: remove

specifies whether the ordering of vehicles in the Vehicles class should be shuffled upon initialization.

spacing : str, optional

specifies the positioning of vehicles in the network relative to one another. May be one of: “uniform”, “random”, or “custom”. Default is “uniform”.

min_gap : float, optional # TODO: remove

minimum gap between two vehicles upon initialization, in meters. Default is 0 m.

x0 : float, optional # TODO: remove

position of the first vehicle to be placed in the network

perturbation : float, optional

standard deviation used to perturb vehicles from their uniform position, in meters. Default is 0 m.

bunching : float, optional

reduces the portion of the network that should be filled with vehicles by this amount.

lanes_distribution : int, optional

number of lanes vehicles should be dispersed into. If the value is greater than the total number of lanes on an edge, vehicles are spread across all lanes.

edges_distribution : str or list of str or dict, optional

edges vehicles may be placed on during initialization, may be one of:

  • “all”: vehicles are distributed over all edges

  • list of edges: list of edges vehicles can be distributed over

  • dict of edges: where the key is the name of the edge to be utilized, and the elements are the number of cars to place on each edge

additional_params : dict, optional

some other network-specific params

class flow.core.params.NetParams(inflows=None, osm_path=None, template=None, additional_params=None)[source]

Bases: object

Network configuration parameters.

Unlike most other parameters, NetParams may vary drastically dependent on the specific network configuration. For example, for the ring road the network parameters will include a characteristic length, number of lanes, and speed limit.

In order to determine which additional_params variable may be needed for a specific network, refer to the ADDITIONAL_NET_PARAMS variable located in the network file.

Attributes

inflows : InFlows type, optional

specifies the inflows of specific edges and the types of vehicles entering the network from these edges

osm_path : str, optional

path to the .osm file that should be used to generate the network configuration files

template : str, optional

path to the network template file that can be used to instantiate a netowrk in the simulator of choice

additional_params : dict, optional

network specific parameters; see each subclass for a description of what is needed

class flow.core.params.SimParams(sim_step=0.1, render=False, restart_instance=False, emission_path=None, save_render=False, sight_radius=25, show_radius=False, pxpm=2, force_color_update=False)[source]

Bases: object

Simulation-specific parameters.

All subsequent parameters of the same type must extend this.

Attributes

sim_step : float optional

seconds per simulation step; 0.1 by default

render : str or bool, optional

specifies whether to visualize the rollout(s)

  • False: no rendering

  • True: delegate rendering to sumo-gui for back-compatibility

  • “gray”: static grayscale rendering, which is good for training

  • “dgray”: dynamic grayscale rendering

  • “rgb”: static RGB rendering

  • “drgb”: dynamic RGB rendering, which is good for visualization

restart_instance : bool, optional

specifies whether to restart a simulation upon reset. Restarting the instance helps avoid slowdowns cause by excessive inflows over large experiment runtimes, but also require the gui to be started after every reset if “render” is set to True.

emission_path : str, optional

Path to the folder in which to create the emissions output. Emissions output is not generated if this value is not specified

save_render : bool, optional

specifies whether to save rendering data to disk

sight_radius : int, optional

sets the radius of observation for RL vehicles (meter)

show_radius : bool, optional

specifies whether to render the radius of RL observation

pxpm : int, optional

specifies rendering resolution (pixel / meter)

force_color_update : bool, optional

whether or not to automatically color vehicles according to their types

class flow.core.params.SumoCarFollowingParams(speed_mode='right_of_way', accel=2.6, decel=4.5, sigma=0.5, tau=1.0, min_gap=2.5, max_speed=30, speed_factor=1.0, speed_dev=0.1, impatience=0.5, car_follow_model='IDM', **kwargs)[source]

Bases: object

Parameters for sumo-controlled acceleration behavior.

Attributes

speed_mode : str or int, optional

may be one of the following:

  • “right_of_way” (default): respect safe speed, right of way and brake hard at red lights if needed. DOES NOT respect max accel and decel which enables emergency stopping. Necessary to prevent custom models from crashing

  • “obey_safe_speed”: prevents vehicles from colliding longitudinally, but can fail in cases where vehicles are allowed to lane change

  • “no_collide”: Human and RL cars are preventing from reaching speeds that may cause crashes (also serves as a failsafe). Note: this may lead to collisions in complex networks

  • “aggressive”: Human and RL cars are not limited by sumo with regard to their accelerations, and can crash longitudinally

  • “all_checks”: all sumo safety checks are activated

  • int values may be used to define custom speed mode for the given vehicles, specified at: http://sumo.dlr.de/wiki/TraCI/Change_Vehicle_State#speed_mode_.280xb3.29

accel : float

see Note

decel : float

see Note

sigma : float

see Note

tau : float

see Note

min_gap : float

see minGap Note

max_speed : float

see maxSpeed Note

speed_factor : float

see speedFactor Note

speed_dev : float

see speedDev in Note

impatience : float

see Note

car_follow_model : str

see carFollowModel in Note

kwargs : dict

used to handle deprecations

class flow.core.params.SumoLaneChangeParams(lane_change_mode='no_lc_safe', model='LC2013', lc_strategic=1.0, lc_cooperative=1.0, lc_speed_gain=1.0, lc_keep_right=1.0, lc_look_ahead_left=2.0, lc_speed_gain_right=1.0, lc_sublane=1.0, lc_pushy=0, lc_pushy_gap=0.6, lc_assertive=1, lc_accel_lat=1.0, **kwargs)[source]

Bases: object

Parameters for sumo-controlled lane change behavior.

Attributes

lane_change_mode : str or int, optional

may be one of the following: * “no_lc_safe” (default): Disable all SUMO lane changing but still

handle safety checks (collision avoidance and safety-gap enforcement) in the simulation. Binary is [001000000000]

  • “no_lc_aggressive”: SUMO lane changes are not executed, collision avoidance and safety-gap enforcement are off. Binary is [000000000000]

  • “sumo_default”: Execute all changes requested by a custom controller unless in conflict with TraCI. Binary is [011001010101].

  • “no_strategic_aggressive”: Execute all changes except strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010001010100]

  • “no_strategic_safe”: Execute all changes except strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011001010100]

  • “only_strategic_aggressive”: Execute only strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000000000001]

  • “only_strategic_safe”: Execute only strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001000000001]

  • “no_cooperative_aggressive”: Execute all changes except cooperative (change in order to allow others to change) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010001010001]

  • “no_cooperative_safe”: Execute all changes except cooperative lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011001010001]

  • “only_cooperative_aggressive”: Execute only cooperative lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000000000100]

  • “only_cooperative_safe”: Execute only cooperative lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001000000100]

  • “no_speed_gain_aggressive”: Execute all changes except speed gain (the

    other lane allows for faster driving) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010001000101]

  • “no_speed_gain_safe”: Execute all changes except speed gain lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011001000101]

  • “only_speed_gain_aggressive”: Execute only speed gain lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000000010000]

  • “only_speed_gain_safe”: Execute only speed gain lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001000010000]

  • “no_right_drive_aggressive”: Execute all changes except right drive (obligation to drive on the right) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010000010101]

  • “no_right_drive_safe”: Execute all changes except right drive lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011000010101]

  • “only_right_drive_aggressive”: Execute only right drive lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000001000000]

  • “only_right_drive_safe”: Execute only right drive lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001001000000]

  • int values may be used to define custom lane change modes for the given vehicles, specified at: http://sumo.dlr.de/wiki/TraCI/Change_Vehicle_State#lane_change_mode_.280xb6.29

model : str, optional

see laneChangeModel in Note

lc_strategic : float, optional

see lcStrategic in Note

lc_cooperative : float, optional

see lcCooperative in Note

lc_speed_gain : float, optional

see lcSpeedGain in Note

lc_keep_right : float, optional

see lcKeepRight in Note

lc_look_ahead_left : float, optional

see lcLookaheadLeft in Note

lc_speed_gain_right : float, optional

see lcSpeedGainRight in Note

lc_sublane : float, optional

see lcSublane in Note

lc_pushy : float, optional

see lcPushy in Note

lc_pushy_gap : float, optional

see lcPushyGap in Note

lc_assertive : float, optional

see lcAssertive in Note

lc_accel_lat : float, optional

see lcAccelLate in Note

kwargs : dict

used to handle deprecations

class flow.core.params.SumoParams(port=None, sim_step=0.1, emission_path=None, lateral_resolution=None, no_step_log=True, render=False, save_render=False, sight_radius=25, show_radius=False, pxpm=2, force_color_update=False, overtake_right=False, seed=None, restart_instance=False, print_warnings=True, teleport_time=- 1, num_clients=1, color_by_speed=False, use_ballistic=False)[source]

Bases: flow.core.params.SimParams

Sumo-specific simulation parameters.

Extends SimParams.

These parameters are used to customize a sumo simulation instance upon initialization. This includes passing the simulation step length, specifying whether to use sumo’s gui during a run, and other features described in the Attributes below.

Attributes

port : int, optional

Port for Traci to connect to; finds an empty port by default

sim_step : float optional

seconds per simulation step; 0.1 by default

emission_path : str, optional

Path to the folder in which to create the emissions output. Emissions output is not generated if this value is not specified

lateral_resolution : float, optional

width of the divided sublanes within a lane, defaults to None (i.e. no sublanes). If this value is specified, the vehicle in the network cannot use the “LC2013” lane change model.

no_step_log : bool, optional

specifies whether to add sumo’s step logs to the log file, and print them into the terminal during runtime, defaults to True

render : str or bool, optional

specifies whether to visualize the rollout(s)

  • False: no rendering

  • True: delegate rendering to sumo-gui for back-compatibility

  • “gray”: static grayscale rendering, which is good for training

  • “dgray”: dynamic grayscale rendering

  • “rgb”: static RGB rendering

  • “drgb”: dynamic RGB rendering, which is good for visualization

save_render : bool, optional

specifies whether to save rendering data to disk

sight_radius : int, optional

sets the radius of observation for RL vehicles (meter)

show_radius : bool, optional

specifies whether to render the radius of RL observation

pxpm : int, optional

specifies rendering resolution (pixel / meter)

force_color_update : bool, optional

whether or not to automatically color vehicles according to their types

overtake_right : bool, optional

whether vehicles are allowed to overtake on the right as well as the left

seed : int, optional

seed for sumo instance

restart_instance : bool, optional

specifies whether to restart a sumo instance upon reset. Restarting the instance helps avoid slowdowns cause by excessive inflows over large experiment runtimes, but also require the gui to be started after every reset if “render” is set to True.

print_warnings : bool, optional

If set to false, this will silence sumo warnings on the stdout

teleport_time : int, optional

If negative, vehicles don’t teleport in gridlock. If positive, they teleport after teleport_time seconds

num_clients : int, optional

Number of clients that will connect to Traci

color_by_speed : bool

whether to color the vehicles by the speed they are moving at the current time step

use_ballistic: bool, optional

If true, use a ballistic integration step instead of an euler step

class flow.core.params.TrafficLightParams(baseline=False)[source]

Bases: object

Base traffic light.

This class is used to place traffic lights in the network and describe the state of these traffic lights. In addition, this class supports modifying the states of certain lights via TraCI.

actuated_default()[source]

Return the default values for an actuated network.

An actuated network is a network for a system where all junctions are actuated traffic lights.

Returns

tl_logic : dict

traffic light logic

add(node_id, tls_type='static', programID=10, offset=None, phases=None, maxGap=None, detectorGap=None, showDetectors=None, file=None, freq=None)[source]

Add a traffic light component to the network.

When generating networks using xml files, using this method to add a traffic light will explicitly place the traffic light in the requested node of the generated network.

If traffic lights are not added here but are already present in the network (e.g. through a prebuilt net.xml file), then the traffic light class will identify and add them separately.

Parameters

node_id : str

name of the node with traffic lights

tls_type : str, optional

type of the traffic light (see Note)

programID : str, optional

id of the traffic light program (see Note)

offset : int, optional

initial time offset of the program

phases : list of dict, optional

list of phases to be followed by the traffic light, defaults to default sumo traffic light behavior. Each element in the list must consist of a dict with two keys:

  • “duration”: length of the current phase cycle (in sec)

  • “state”: string consist the sequence of states in the phase

  • “minDur”: optional

    The minimum duration of the phase when using type actuated

  • “maxDur”: optional

    The maximum duration of the phase when using type actuated

maxGap : int, optional

describes the maximum time gap between successive vehicle that will cause the current phase to be prolonged, used for actuated traffic lights

detectorGap : int, optional

used for actuated traffic lights determines the time distance between the (automatically generated) detector and the stop line in seconds (at each lanes maximum speed), used for actuated traffic lights

showDetectors : bool, optional

toggles whether or not detectors are shown in sumo-gui, used for actuated traffic lights

file : str, optional

which file the detector shall write results into

freq : int, optional

the period over which collected values shall be aggregated

get_properties()[source]

Return traffic light properties.

This is meant to be used by the generator to import traffic light data to the .net.xml file

class flow.core.params.VehicleParams[source]

Bases: object

Base vehicle class.

This is used to describe the state of all vehicles in the network. State information on the vehicles for a given time step can be set or retrieved from this class.

add(veh_id, acceleration_controller=(<class 'flow.controllers.car_following_models.SimCarFollowingController'>, {}), lane_change_controller=(<class 'flow.controllers.lane_change_controllers.SimLaneChangeController'>, {}), routing_controller=None, initial_speed=0, num_vehicles=0, car_following_params=None, lane_change_params=None, color=None)[source]

Add a sequence of vehicles to the list of vehicles in the network.

Parameters

veh_id : str

base vehicle ID for the vehicles (will be appended by a number)

acceleration_controller : tup, optional

1st element: flow-specified acceleration controller 2nd element: controller parameters (may be set to None to maintain default parameters)

lane_change_controller : tup, optional

1st element: flow-specified lane-changer controller 2nd element: controller parameters (may be set to None to maintain default parameters)

routing_controller : tup, optional

1st element: flow-specified routing controller 2nd element: controller parameters (may be set to None to maintain default parameters)

initial_speed : float, optional

initial speed of the vehicles being added (in m/s)

num_vehicles : int, optional

number of vehicles of this type to be added to the network

car_following_params : flow.core.params.SumoCarFollowingParams

Params object specifying attributes for Sumo car following model.

lane_change_params : flow.core.params.SumoLaneChangeParams

Params object specifying attributes for Sumo lane changing model.

get_type(veh_id)[source]

Return the type of a specified vehicle.

Parameters

veh_id : str

vehicle ID whose type the user is querying

initial

list : initial state of the vehicles class, used for serialization purposes

minGap

dict (str, int) : contains the minGap attribute of each type of vehicle

num_rl_vehicles

int : number of rl vehicles in the network

num_types

int : number of unique types of vehicles in the network

num_vehicles

total number of vehicles in the network

type_parameters

dict (str, str) : contains the parameters associated with each type of vehicle

types

list of str : types of vehicles in the network

flow.core.rewards module

A series of reward functions.

flow.core.rewards.average_velocity(env, fail=False)[source]

Encourage proximity to an average velocity.

This reward function returns the average velocity of all vehicles in the system.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

fail : bool, optional

specifies if any crash or other failure occurred in the system

Returns

float

reward value

flow.core.rewards.avg_delay_specified_vehicles(env, veh_ids)[source]

Calculate the average delay for a set of vehicles in the system.

Parameters

env: flow.envs.Env

the environment variable, which contains information on the current state of the system.

veh_ids: a list of the ids of the vehicles, for which we are calculating

average delay

Returns
——-

float

average delay

flow.core.rewards.boolean_action_penalty(discrete_actions, gain=1.0)[source]

Penalize boolean actions that indicate a switch.

flow.core.rewards.desired_velocity(env, fail=False, edge_list=None)[source]

Encourage proximity to a desired velocity.

This function measures the deviation of a system of vehicles from a user-specified desired velocity peaking when all vehicles in the ring are set to this desired velocity. Moreover, in order to ensure that the reward function naturally punishing the early termination of rollouts due to collisions or other failures, the function is formulated as a mapping \(r: \\mathcal{S} \\times \\mathcal{A} \\rightarrow \\mathbb{R}_{\\geq 0}\). This is done by subtracting the deviation of the system from the desired velocity from the peak allowable deviation from the desired velocity. Additionally, since the velocity of vehicles are unbounded above, the reward is bounded below by zero, to ensure nonnegativity.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

fail : bool, optional

specifies if any crash or other failure occurred in the system

edge_list : list of str, optional

list of edges the reward is computed over. If no edge_list is defined, the reward is computed over all edges

Returns

float

reward value

flow.core.rewards.energy_consumption(env, gain=0.001)[source]

Calculate power consumption of a vehicle.

Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.

flow.core.rewards.miles_per_gallon(env, veh_ids=None, gain=0.001)[source]

Calculate mpg of either a particular vehicle or the total average of all the vehicles.

Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

veh_ids : [list]

list of veh_ids to compute the reward over

gain : float

scaling factor for the reward

flow.core.rewards.miles_per_megajoule(env, veh_ids=None, gain=0.001)[source]

Calculate miles per mega-joule of either a particular vehicle or the total average of all the vehicles.

Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

veh_ids : [list]

list of veh_ids to compute the reward over

gain : float

scaling factor for the reward

flow.core.rewards.min_delay(env)[source]

Reward function used to encourage minimization of total delay.

This function measures the deviation of a system of vehicles from all the vehicles smoothly travelling at a fixed speed to their destinations.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

Returns

float

reward value

flow.core.rewards.min_delay_unscaled(env)[source]

Return the average delay for all vehicles in the system.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

Returns

float

reward value

flow.core.rewards.penalize_headway_variance(vehicles, vids, normalization=1, penalty_gain=1, penalty_exponent=1)[source]

Reward function used to train rl vehicles to encourage large headways.

Parameters

vehicles : flow.core.kernel.vehicle.KernelVehicle

contains the state of all vehicles in the network (generally self.vehicles)

vids : list of str

list of ids for vehicles

normalization : float, optional

constant for scaling (down) the headways

penalty_gain : float, optional

sets the penalty for each vehicle between 0 and this value

penalty_exponent : float, optional

used to allow exponential punishing of smaller headways

flow.core.rewards.penalize_near_standstill(env, thresh=0.3, gain=1)[source]

Reward function which penalizes vehicles at a low velocity.

This reward function is used to penalize vehicles below a specified threshold. This assists with discouraging RL from gamifying a network, which can result in standstill behavior or similarly bad, near-zero velocities.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current

thresh : float

the velocity threshold below which penalties are applied

gain : float

multiplicative factor on the action penalty

flow.core.rewards.penalize_standstill(env, gain=1)[source]

Reward function that penalizes vehicle standstill.

Is it better for this to be:
  1. penalize standstill in general?

  2. multiplicative based on time that vel=0?

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

gain : float

multiplicative factor on the action penalty

Returns

float

reward value

flow.core.rewards.punish_rl_lane_changes(env, penalty=1)[source]

Penalize an RL vehicle performing lane changes.

This reward function is meant to minimize the number of lane changes and RL vehicle performs.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

penalty : float, optional

penalty imposed on the reward function for any rl lane change action

flow.core.rewards.rl_forward_progress(env, gain=0.1)[source]

Rewared function used to reward the RL vehicles for travelling forward.

Parameters

env : flow.envs.Env

the environment variable, which contains information on the current state of the system.

gain : float

specifies how much to reward the RL vehicles

Returns

float

reward value

flow.core.rewards.veh_energy_consumption(env, veh_id, gain=0.001)[source]

Calculate power consumption of a vehicle.

Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.

flow.core.util module

A collection of utility functions for Flow.

flow.core.util.emission_to_csv(emission_path, output_path=None)[source]

Convert an emission file generated by sumo into a csv file.

Note that the emission file contains information generated by sumo, not flow. This means that some data, such as absolute position, is not immediately available from the emission file, but can be recreated.

Parameters

emission_path : str

path to the emission file that should be converted

output_path : str

path to the csv file that will be generated, default is the same directory as the emission file, with the same name

flow.core.util.ensure_dir(path)[source]

Ensure that the directory specified exists, and if not, create it.

flow.core.util.makexml(name, nsl)[source]

Create an xml file.

flow.core.util.printxml(t, fn)[source]

Print information from a dict into an xml file.

Module contents

Empty init file to ensure documentation for core is created.