flow.core package¶
Subpackages¶
Submodules¶
flow.core.experiment module¶
flow.core.params module¶
Objects that define the various meta-parameters of an experiment.
-
class
flow.core.params.
AimsunParams
(sim_step=0.1, render=False, restart_instance=False, emission_path=None, save_render=False, sight_radius=25, show_radius=False, pxpm=2, network_name='Dynamic Scenario 866', experiment_name='Micro SRC Experiment 867', replication_name='Replication 870', centroid_config_name=None, subnetwork_name=None)[source]¶ Bases:
flow.core.params.SimParams
Aimsun-specific simulation parameters.
Extends SimParams.
- Attributes
sim_step : float optional
seconds per simulation step; 0.1 by default
render : str or bool, optional
specifies whether to visualize the rollout(s)
False: no rendering
True: delegate rendering to sumo-gui for back-compatibility
“gray”: static grayscale rendering, which is good for training
“dgray”: dynamic grayscale rendering
“rgb”: static RGB rendering
“drgb”: dynamic RGB rendering, which is good for visualization
restart_instance : bool, optional
specifies whether to restart a simulation upon reset. Restarting the instance helps avoid slowdowns cause by excessive inflows over large experiment runtimes, but also require the gui to be started after every reset if “render” is set to True.
emission_path : str, optional
Path to the folder in which to create the emissions output. Emissions output is not generated if this value is not specified
save_render : bool, optional
specifies whether to save rendering data to disk
sight_radius : int, optional
sets the radius of observation for RL vehicles (meter)
show_radius : bool, optional
specifies whether to render the radius of RL observation
pxpm : int, optional
specifies rendering resolution (pixel / meter)
network_name : str, optional
name of the network generated in Aimsun.
experiment_name : str, optional
name of the experiment generated in Aimsun
replication_name : str, optional
name of the replication generated in Aimsun. When loading an Aimsun template, this parameter must be set to the name of the replication to be run by the simulation; in this case, the network_name and experiment_name parameters are not necessary as they will be obtained from the replication name.
centroid_config_name : str, optional
name of the centroid configuration to load in Aimsun. This parameter is only used when loading an Aimsun template, not when generating one.
subnetwork_name : str, optional
name of the subnetwork to load in Aimsun. This parameter is not used when generating a network; it can be used when loading an Aimsun template containing a subnetwork in order to only load the objects contained in this subnetwork. If set to None or if the specified subnetwork does not exist, the whole network will be loaded.
-
class
flow.core.params.
EnvParams
(additional_params=None, horizon=inf, warmup_steps=0, sims_per_step=1, evaluate=False, clip_actions=True)[source]¶ Bases:
object
Environment and experiment-specific parameters.
This includes specifying the bounds of the action space and relevant coefficients to the reward function, as well as specifying how the positions of vehicles are modified in between rollouts.
- Attributes
additional_params : dict, optional
Specify additional environment params for a specific environment configuration
horizon : int, optional
number of steps per rollouts
warmup_steps : int, optional
number of steps performed before the initialization of training during a rollout. These warmup steps are not added as steps into training, and the actions of rl agents during these steps are dictated by sumo. Defaults to zero
sims_per_step : int, optional
number of sumo simulation steps performed in any given rollout step. RL agents perform the same action for the duration of these simulation steps.
evaluate : bool, optional
flag indicating that the evaluation reward should be used so the evaluation reward should be used rather than the normal reward
clip_actions : bool, optional
specifies whether to clip actions from the policy by their range when they are inputted to the reward function. Note that the actions are still clipped before they are provided to apply_rl_actions.
-
class
flow.core.params.
InFlows
[source]¶ Bases:
object
Used to add inflows to a network.
Inflows can be specified for any edge that has a specified route or routes.
-
add
(edge, veh_type, vehs_per_hour=None, probability=None, period=None, depart_lane='first', depart_speed=0, name='flow', begin=1, end=86400, number=None, **kwargs)[source]¶ Specify a new inflow for a given type of vehicles and edge.
- Parameters
edge : str
starting edge for the vehicles in this inflow
veh_type : str
type of the vehicles entering the edge. Must match one of the types set in the Vehicles class
vehs_per_hour : float, optional
number of vehicles per hour, equally spaced (in vehicles/hour). Cannot be specified together with probability or period
probability : float, optional
probability for emitting a vehicle each second (between 0 and 1). Cannot be specified together with vehs_per_hour or period
period : float, optional
insert equally spaced vehicles at that period (in seconds). Cannot be specified together with vehs_per_hour or probability
depart_lane : int or str
the lane on which the vehicle shall be inserted. Can be either one of:
int >= 0: index of the lane (starting with rightmost = 0)
“random”: a random lane is chosen, but the vehicle insertion is not retried if it could not be inserted
“free”: the most free (least occupied) lane is chosen
“best”: the “free” lane (see above) among those who allow the vehicle the longest ride without the need to change lane
“first”: the rightmost lane the vehicle may use
Defaults to “first”.
depart_speed : float or str
the speed with which the vehicle shall enter the network (in m/s) can be either one of:
float >= 0: the vehicle is tried to be inserted using the given speed; if that speed is unsafe, departure is delayed
“random”: vehicles enter the edge with a random speed between 0 and the speed limit on the edge; the entering speed may be adapted to ensure a safe distance to the leading vehicle is kept
“speedLimit”: vehicles enter the edge with the maximum speed that is allowed on this edge; if that speed is unsafe, departure is delayed
Defaults to 0.
name : str, optional
prefix for the id of the vehicles entering via this inflow. Defaults to “flow”
begin : float, optional
first vehicle departure time (in seconds, minimum 1 second). Defaults to 1 second
end : float, optional
end of departure interval (in seconds). This parameter is not taken into account if ‘number’ is specified. Defaults to 24 hours
number : int, optional
total number of vehicles the inflow should create (due to rounding up, this parameter may not be exactly enforced and shouldn’t be set too small). Default: infinite (c.f. ‘end’ parameter)
kwargs : dict, optional
see Note
-
-
class
flow.core.params.
InitialConfig
(shuffle=False, spacing='uniform', min_gap=0, perturbation=0.0, x0=0, bunching=0, lanes_distribution=inf, edges_distribution='all', additional_params=None)[source]¶ Bases:
object
Initial configuration parameters.
These parameters that affect the positioning of vehicle in the network at the start of a rollout. By default, vehicles are uniformly distributed in the network.
- Attributes
shuffle : bool, optional # TODO: remove
specifies whether the ordering of vehicles in the Vehicles class should be shuffled upon initialization.
spacing : str, optional
specifies the positioning of vehicles in the network relative to one another. May be one of: “uniform”, “random”, or “custom”. Default is “uniform”.
min_gap : float, optional # TODO: remove
minimum gap between two vehicles upon initialization, in meters. Default is 0 m.
x0 : float, optional # TODO: remove
position of the first vehicle to be placed in the network
perturbation : float, optional
standard deviation used to perturb vehicles from their uniform position, in meters. Default is 0 m.
bunching : float, optional
reduces the portion of the network that should be filled with vehicles by this amount.
lanes_distribution : int, optional
number of lanes vehicles should be dispersed into. If the value is greater than the total number of lanes on an edge, vehicles are spread across all lanes.
edges_distribution : str or list of str or dict, optional
edges vehicles may be placed on during initialization, may be one of:
“all”: vehicles are distributed over all edges
list of edges: list of edges vehicles can be distributed over
dict of edges: where the key is the name of the edge to be utilized, and the elements are the number of cars to place on each edge
additional_params : dict, optional
some other network-specific params
-
class
flow.core.params.
NetParams
(inflows=None, osm_path=None, template=None, additional_params=None)[source]¶ Bases:
object
Network configuration parameters.
Unlike most other parameters, NetParams may vary drastically dependent on the specific network configuration. For example, for the ring road the network parameters will include a characteristic length, number of lanes, and speed limit.
In order to determine which additional_params variable may be needed for a specific network, refer to the ADDITIONAL_NET_PARAMS variable located in the network file.
- Attributes
inflows : InFlows type, optional
specifies the inflows of specific edges and the types of vehicles entering the network from these edges
osm_path : str, optional
path to the .osm file that should be used to generate the network configuration files
template : str, optional
path to the network template file that can be used to instantiate a netowrk in the simulator of choice
additional_params : dict, optional
network specific parameters; see each subclass for a description of what is needed
-
class
flow.core.params.
SimParams
(sim_step=0.1, render=False, restart_instance=False, emission_path=None, save_render=False, sight_radius=25, show_radius=False, pxpm=2, force_color_update=False)[source]¶ Bases:
object
Simulation-specific parameters.
All subsequent parameters of the same type must extend this.
- Attributes
sim_step : float optional
seconds per simulation step; 0.1 by default
render : str or bool, optional
specifies whether to visualize the rollout(s)
False: no rendering
True: delegate rendering to sumo-gui for back-compatibility
“gray”: static grayscale rendering, which is good for training
“dgray”: dynamic grayscale rendering
“rgb”: static RGB rendering
“drgb”: dynamic RGB rendering, which is good for visualization
restart_instance : bool, optional
specifies whether to restart a simulation upon reset. Restarting the instance helps avoid slowdowns cause by excessive inflows over large experiment runtimes, but also require the gui to be started after every reset if “render” is set to True.
emission_path : str, optional
Path to the folder in which to create the emissions output. Emissions output is not generated if this value is not specified
save_render : bool, optional
specifies whether to save rendering data to disk
sight_radius : int, optional
sets the radius of observation for RL vehicles (meter)
show_radius : bool, optional
specifies whether to render the radius of RL observation
pxpm : int, optional
specifies rendering resolution (pixel / meter)
force_color_update : bool, optional
whether or not to automatically color vehicles according to their types
-
class
flow.core.params.
SumoCarFollowingParams
(speed_mode='right_of_way', accel=2.6, decel=4.5, sigma=0.5, tau=1.0, min_gap=2.5, max_speed=30, speed_factor=1.0, speed_dev=0.1, impatience=0.5, car_follow_model='IDM', **kwargs)[source]¶ Bases:
object
Parameters for sumo-controlled acceleration behavior.
- Attributes
speed_mode : str or int, optional
may be one of the following:
“right_of_way” (default): respect safe speed, right of way and brake hard at red lights if needed. DOES NOT respect max accel and decel which enables emergency stopping. Necessary to prevent custom models from crashing
“obey_safe_speed”: prevents vehicles from colliding longitudinally, but can fail in cases where vehicles are allowed to lane change
“no_collide”: Human and RL cars are preventing from reaching speeds that may cause crashes (also serves as a failsafe). Note: this may lead to collisions in complex networks
“aggressive”: Human and RL cars are not limited by sumo with regard to their accelerations, and can crash longitudinally
“all_checks”: all sumo safety checks are activated
int values may be used to define custom speed mode for the given vehicles, specified at: http://sumo.dlr.de/wiki/TraCI/Change_Vehicle_State#speed_mode_.280xb3.29
accel : float
see Note
decel : float
see Note
sigma : float
see Note
tau : float
see Note
min_gap : float
see minGap Note
max_speed : float
see maxSpeed Note
speed_factor : float
see speedFactor Note
speed_dev : float
see speedDev in Note
impatience : float
see Note
car_follow_model : str
see carFollowModel in Note
kwargs : dict
used to handle deprecations
-
class
flow.core.params.
SumoLaneChangeParams
(lane_change_mode='no_lc_safe', model='LC2013', lc_strategic=1.0, lc_cooperative=1.0, lc_speed_gain=1.0, lc_keep_right=1.0, lc_look_ahead_left=2.0, lc_speed_gain_right=1.0, lc_sublane=1.0, lc_pushy=0, lc_pushy_gap=0.6, lc_assertive=1, lc_accel_lat=1.0, **kwargs)[source]¶ Bases:
object
Parameters for sumo-controlled lane change behavior.
- Attributes
lane_change_mode : str or int, optional
may be one of the following: * “no_lc_safe” (default): Disable all SUMO lane changing but still
handle safety checks (collision avoidance and safety-gap enforcement) in the simulation. Binary is [001000000000]
“no_lc_aggressive”: SUMO lane changes are not executed, collision avoidance and safety-gap enforcement are off. Binary is [000000000000]
“sumo_default”: Execute all changes requested by a custom controller unless in conflict with TraCI. Binary is [011001010101].
“no_strategic_aggressive”: Execute all changes except strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010001010100]
“no_strategic_safe”: Execute all changes except strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011001010100]
“only_strategic_aggressive”: Execute only strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000000000001]
“only_strategic_safe”: Execute only strategic (routing) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001000000001]
“no_cooperative_aggressive”: Execute all changes except cooperative (change in order to allow others to change) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010001010001]
“no_cooperative_safe”: Execute all changes except cooperative lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011001010001]
“only_cooperative_aggressive”: Execute only cooperative lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000000000100]
“only_cooperative_safe”: Execute only cooperative lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001000000100]
- “no_speed_gain_aggressive”: Execute all changes except speed gain (the
other lane allows for faster driving) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010001000101]
“no_speed_gain_safe”: Execute all changes except speed gain lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011001000101]
“only_speed_gain_aggressive”: Execute only speed gain lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000000010000]
“only_speed_gain_safe”: Execute only speed gain lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001000010000]
“no_right_drive_aggressive”: Execute all changes except right drive (obligation to drive on the right) lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [010000010101]
“no_right_drive_safe”: Execute all changes except right drive lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [011000010101]
“only_right_drive_aggressive”: Execute only right drive lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are off. Binary is [000001000000]
“only_right_drive_safe”: Execute only right drive lane changes unless in conflict with TraCI. Collision avoidance and safety-gap enforcement are on. Binary is [001001000000]
int values may be used to define custom lane change modes for the given vehicles, specified at: http://sumo.dlr.de/wiki/TraCI/Change_Vehicle_State#lane_change_mode_.280xb6.29
model : str, optional
see laneChangeModel in Note
lc_strategic : float, optional
see lcStrategic in Note
lc_cooperative : float, optional
see lcCooperative in Note
lc_speed_gain : float, optional
see lcSpeedGain in Note
lc_keep_right : float, optional
see lcKeepRight in Note
lc_look_ahead_left : float, optional
see lcLookaheadLeft in Note
lc_speed_gain_right : float, optional
see lcSpeedGainRight in Note
lc_sublane : float, optional
see lcSublane in Note
lc_pushy : float, optional
see lcPushy in Note
lc_pushy_gap : float, optional
see lcPushyGap in Note
lc_assertive : float, optional
see lcAssertive in Note
lc_accel_lat : float, optional
see lcAccelLate in Note
kwargs : dict
used to handle deprecations
-
class
flow.core.params.
SumoParams
(port=None, sim_step=0.1, emission_path=None, lateral_resolution=None, no_step_log=True, render=False, save_render=False, sight_radius=25, show_radius=False, pxpm=2, force_color_update=False, overtake_right=False, seed=None, restart_instance=False, print_warnings=True, teleport_time=- 1, num_clients=1, color_by_speed=False, use_ballistic=False)[source]¶ Bases:
flow.core.params.SimParams
Sumo-specific simulation parameters.
Extends SimParams.
These parameters are used to customize a sumo simulation instance upon initialization. This includes passing the simulation step length, specifying whether to use sumo’s gui during a run, and other features described in the Attributes below.
- Attributes
port : int, optional
Port for Traci to connect to; finds an empty port by default
sim_step : float optional
seconds per simulation step; 0.1 by default
emission_path : str, optional
Path to the folder in which to create the emissions output. Emissions output is not generated if this value is not specified
lateral_resolution : float, optional
width of the divided sublanes within a lane, defaults to None (i.e. no sublanes). If this value is specified, the vehicle in the network cannot use the “LC2013” lane change model.
no_step_log : bool, optional
specifies whether to add sumo’s step logs to the log file, and print them into the terminal during runtime, defaults to True
render : str or bool, optional
specifies whether to visualize the rollout(s)
False: no rendering
True: delegate rendering to sumo-gui for back-compatibility
“gray”: static grayscale rendering, which is good for training
“dgray”: dynamic grayscale rendering
“rgb”: static RGB rendering
“drgb”: dynamic RGB rendering, which is good for visualization
save_render : bool, optional
specifies whether to save rendering data to disk
sight_radius : int, optional
sets the radius of observation for RL vehicles (meter)
show_radius : bool, optional
specifies whether to render the radius of RL observation
pxpm : int, optional
specifies rendering resolution (pixel / meter)
force_color_update : bool, optional
whether or not to automatically color vehicles according to their types
overtake_right : bool, optional
whether vehicles are allowed to overtake on the right as well as the left
seed : int, optional
seed for sumo instance
restart_instance : bool, optional
specifies whether to restart a sumo instance upon reset. Restarting the instance helps avoid slowdowns cause by excessive inflows over large experiment runtimes, but also require the gui to be started after every reset if “render” is set to True.
print_warnings : bool, optional
If set to false, this will silence sumo warnings on the stdout
teleport_time : int, optional
If negative, vehicles don’t teleport in gridlock. If positive, they teleport after teleport_time seconds
num_clients : int, optional
Number of clients that will connect to Traci
color_by_speed : bool
whether to color the vehicles by the speed they are moving at the current time step
use_ballistic: bool, optional
If true, use a ballistic integration step instead of an euler step
-
class
flow.core.params.
TrafficLightParams
(baseline=False)[source]¶ Bases:
object
Base traffic light.
This class is used to place traffic lights in the network and describe the state of these traffic lights. In addition, this class supports modifying the states of certain lights via TraCI.
-
actuated_default
()[source]¶ Return the default values for an actuated network.
An actuated network is a network for a system where all junctions are actuated traffic lights.
- Returns
tl_logic : dict
traffic light logic
-
add
(node_id, tls_type='static', programID=10, offset=None, phases=None, maxGap=None, detectorGap=None, showDetectors=None, file=None, freq=None)[source]¶ Add a traffic light component to the network.
When generating networks using xml files, using this method to add a traffic light will explicitly place the traffic light in the requested node of the generated network.
If traffic lights are not added here but are already present in the network (e.g. through a prebuilt net.xml file), then the traffic light class will identify and add them separately.
- Parameters
node_id : str
name of the node with traffic lights
tls_type : str, optional
type of the traffic light (see Note)
programID : str, optional
id of the traffic light program (see Note)
offset : int, optional
initial time offset of the program
phases : list of dict, optional
list of phases to be followed by the traffic light, defaults to default sumo traffic light behavior. Each element in the list must consist of a dict with two keys:
“duration”: length of the current phase cycle (in sec)
“state”: string consist the sequence of states in the phase
- “minDur”: optional
The minimum duration of the phase when using type actuated
- “maxDur”: optional
The maximum duration of the phase when using type actuated
maxGap : int, optional
describes the maximum time gap between successive vehicle that will cause the current phase to be prolonged, used for actuated traffic lights
detectorGap : int, optional
used for actuated traffic lights determines the time distance between the (automatically generated) detector and the stop line in seconds (at each lanes maximum speed), used for actuated traffic lights
showDetectors : bool, optional
toggles whether or not detectors are shown in sumo-gui, used for actuated traffic lights
file : str, optional
which file the detector shall write results into
freq : int, optional
the period over which collected values shall be aggregated
-
-
class
flow.core.params.
VehicleParams
[source]¶ Bases:
object
Base vehicle class.
This is used to describe the state of all vehicles in the network. State information on the vehicles for a given time step can be set or retrieved from this class.
-
add
(veh_id, acceleration_controller=(<class 'flow.controllers.car_following_models.SimCarFollowingController'>, {}), lane_change_controller=(<class 'flow.controllers.lane_change_controllers.SimLaneChangeController'>, {}), routing_controller=None, initial_speed=0, num_vehicles=0, car_following_params=None, lane_change_params=None, color=None)[source]¶ Add a sequence of vehicles to the list of vehicles in the network.
- Parameters
veh_id : str
base vehicle ID for the vehicles (will be appended by a number)
acceleration_controller : tup, optional
1st element: flow-specified acceleration controller 2nd element: controller parameters (may be set to None to maintain default parameters)
lane_change_controller : tup, optional
1st element: flow-specified lane-changer controller 2nd element: controller parameters (may be set to None to maintain default parameters)
routing_controller : tup, optional
1st element: flow-specified routing controller 2nd element: controller parameters (may be set to None to maintain default parameters)
initial_speed : float, optional
initial speed of the vehicles being added (in m/s)
num_vehicles : int, optional
number of vehicles of this type to be added to the network
car_following_params : flow.core.params.SumoCarFollowingParams
Params object specifying attributes for Sumo car following model.
lane_change_params : flow.core.params.SumoLaneChangeParams
Params object specifying attributes for Sumo lane changing model.
-
get_type
(veh_id)[source]¶ Return the type of a specified vehicle.
- Parameters
veh_id : str
vehicle ID whose type the user is querying
-
initial
¶ list : initial state of the vehicles class, used for serialization purposes
-
minGap
¶ dict (str, int) : contains the minGap attribute of each type of vehicle
-
num_rl_vehicles
¶ int : number of rl vehicles in the network
-
num_types
¶ int : number of unique types of vehicles in the network
-
num_vehicles
¶ total number of vehicles in the network
-
type_parameters
¶ dict (str, str) : contains the parameters associated with each type of vehicle
-
types
¶ list of str : types of vehicles in the network
-
flow.core.rewards module¶
A series of reward functions.
-
flow.core.rewards.
average_velocity
(env, fail=False)[source]¶ Encourage proximity to an average velocity.
This reward function returns the average velocity of all vehicles in the system.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
fail : bool, optional
specifies if any crash or other failure occurred in the system
- Returns
float
reward value
-
flow.core.rewards.
avg_delay_specified_vehicles
(env, veh_ids)[source]¶ Calculate the average delay for a set of vehicles in the system.
- Parameters
env: flow.envs.Env
the environment variable, which contains information on the current state of the system.
veh_ids: a list of the ids of the vehicles, for which we are calculating
average delay
- Returns
- ——-
float
average delay
-
flow.core.rewards.
boolean_action_penalty
(discrete_actions, gain=1.0)[source]¶ Penalize boolean actions that indicate a switch.
-
flow.core.rewards.
desired_velocity
(env, fail=False, edge_list=None)[source]¶ Encourage proximity to a desired velocity.
This function measures the deviation of a system of vehicles from a user-specified desired velocity peaking when all vehicles in the ring are set to this desired velocity. Moreover, in order to ensure that the reward function naturally punishing the early termination of rollouts due to collisions or other failures, the function is formulated as a mapping \(r: \\mathcal{S} \\times \\mathcal{A} \\rightarrow \\mathbb{R}_{\\geq 0}\). This is done by subtracting the deviation of the system from the desired velocity from the peak allowable deviation from the desired velocity. Additionally, since the velocity of vehicles are unbounded above, the reward is bounded below by zero, to ensure nonnegativity.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
fail : bool, optional
specifies if any crash or other failure occurred in the system
edge_list : list of str, optional
list of edges the reward is computed over. If no edge_list is defined, the reward is computed over all edges
- Returns
float
reward value
-
flow.core.rewards.
energy_consumption
(env, gain=0.001)[source]¶ Calculate power consumption of a vehicle.
Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.
-
flow.core.rewards.
miles_per_gallon
(env, veh_ids=None, gain=0.001)[source]¶ Calculate mpg of either a particular vehicle or the total average of all the vehicles.
Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
veh_ids : [list]
list of veh_ids to compute the reward over
gain : float
scaling factor for the reward
-
flow.core.rewards.
miles_per_megajoule
(env, veh_ids=None, gain=0.001)[source]¶ Calculate miles per mega-joule of either a particular vehicle or the total average of all the vehicles.
Assumes vehicle is an average sized vehicle. The power calculated here is the lower bound of the actual power consumed by a vehicle.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
veh_ids : [list]
list of veh_ids to compute the reward over
gain : float
scaling factor for the reward
-
flow.core.rewards.
min_delay
(env)[source]¶ Reward function used to encourage minimization of total delay.
This function measures the deviation of a system of vehicles from all the vehicles smoothly travelling at a fixed speed to their destinations.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
- Returns
float
reward value
-
flow.core.rewards.
min_delay_unscaled
(env)[source]¶ Return the average delay for all vehicles in the system.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
- Returns
float
reward value
-
flow.core.rewards.
penalize_headway_variance
(vehicles, vids, normalization=1, penalty_gain=1, penalty_exponent=1)[source]¶ Reward function used to train rl vehicles to encourage large headways.
- Parameters
vehicles : flow.core.kernel.vehicle.KernelVehicle
contains the state of all vehicles in the network (generally self.vehicles)
vids : list of str
list of ids for vehicles
normalization : float, optional
constant for scaling (down) the headways
penalty_gain : float, optional
sets the penalty for each vehicle between 0 and this value
penalty_exponent : float, optional
used to allow exponential punishing of smaller headways
-
flow.core.rewards.
penalize_near_standstill
(env, thresh=0.3, gain=1)[source]¶ Reward function which penalizes vehicles at a low velocity.
This reward function is used to penalize vehicles below a specified threshold. This assists with discouraging RL from gamifying a network, which can result in standstill behavior or similarly bad, near-zero velocities.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current
thresh : float
the velocity threshold below which penalties are applied
gain : float
multiplicative factor on the action penalty
-
flow.core.rewards.
penalize_standstill
(env, gain=1)[source]¶ Reward function that penalizes vehicle standstill.
- Is it better for this to be:
penalize standstill in general?
multiplicative based on time that vel=0?
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
gain : float
multiplicative factor on the action penalty
- Returns
float
reward value
-
flow.core.rewards.
punish_rl_lane_changes
(env, penalty=1)[source]¶ Penalize an RL vehicle performing lane changes.
This reward function is meant to minimize the number of lane changes and RL vehicle performs.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
penalty : float, optional
penalty imposed on the reward function for any rl lane change action
-
flow.core.rewards.
rl_forward_progress
(env, gain=0.1)[source]¶ Rewared function used to reward the RL vehicles for travelling forward.
- Parameters
env : flow.envs.Env
the environment variable, which contains information on the current state of the system.
gain : float
specifies how much to reward the RL vehicles
- Returns
float
reward value
flow.core.util module¶
A collection of utility functions for Flow.
-
flow.core.util.
emission_to_csv
(emission_path, output_path=None)[source]¶ Convert an emission file generated by sumo into a csv file.
Note that the emission file contains information generated by sumo, not flow. This means that some data, such as absolute position, is not immediately available from the emission file, but can be recreated.
- Parameters
emission_path : str
path to the emission file that should be converted
output_path : str
path to the csv file that will be generated, default is the same directory as the emission file, with the same name
Module contents¶
Empty init file to ensure documentation for core is created.