Torque Humanoid

class HumanoidTorque(**kwargs)

Bases: BaseHumanoid

Description

MuJoCo simulation of a humanoid model with one torque actuator per joint.

Tasks

• Walking: The robot has to walk forward with a fixed speed of 1.25 m/s.

• Running: Run forward with a fixed speed of 2.5 m/s.

Dataset Types

The available dataset types for this environment can be found at: Overview of Environments, Tasks and Datasets.

Observation Space

The observation space has the following properties by default (i.e., only obs with Disabled == False):

For walking and running task: (min=-inf, max=inf, dim=36, dtype=float32)

Some observations are disabled by default, but can be turned on. The detailed observation space is:

Index	Description	Min	Max	Disabled	Dim	Units
0	Position of Joint pelvis_ty	-inf	inf	False	1	Position [m]
1	Position of Joint pelvis_tilt	-inf	inf	False	1	Angle [rad]
2	Position of Joint pelvis_list	-inf	inf	False	1	Angle [rad]
3	Position of Joint pelvis_rotation	-inf	inf	False	1	Angle [rad]
4	Position of Joint hip_flexion_r	-0.787	0.787	False	1	Angle [rad]
5	Position of Joint hip_adduction_r	-0.524	0.524	False	1	Angle [rad]
6	Position of Joint hip_rotation_r	-2.0944	2.0944	False	1	Angle [rad]
7	Position of Joint knee_angle_r	-2.0944	0.174533	False	1	Angle [rad]
8	Position of Joint ankle_angle_r	-1.5708	1.5708	False	1	Angle [rad]
9	Position of Joint hip_flexion_l	-0.787	0.787	False	1	Angle [rad]
10	Position of Joint hip_adduction_l	-0.524	0.524	False	1	Angle [rad]
11	Position of Joint hip_rotation_l	-2.0944	2.0944	False	1	Angle [rad]
12	Position of Joint knee_angle_l	-2.0944	0.174533	False	1	Angle [rad]
13	Position of Joint ankle_angle_l	-1.0472	1.0472	False	1	Angle [rad]
14	Position of Joint lumbar_extension	-1.5708	0.377	False	1	Angle [rad]
15	Position of Joint lumbar_bending	-0.754	0.754	False	1	Angle [rad]
16	Position of Joint lumbar_rotation	-0.754	0.754	False	1	Angle [rad]
17	Position of Joint arm_flex_r	-1.5708	1.5708	True	1	Angle [rad]
18	Position of Joint arm_add_r	-2.0944	1.5708	True	1	Angle [rad]
19	Position of Joint arm_rot_r	-1.5708	1.5708	True	1	Angle [rad]
20	Position of Joint elbow_flex_r	-0.3	2.618	True	1	Angle [rad]
21	Position of Joint pro_sup_r	-0.6	1.5708	True	1	Angle [rad]
22	Position of Joint wrist_flex_r	-1.22173	1.22173	True	1	Angle [rad]
23	Position of Joint wrist_dev_r	-0.436332	0.610865	True	1	Angle [rad]
24	Position of Joint arm_flex_l	-1.5708	1.5708	True	1	Angle [rad]
25	Position of Joint arm_add_l	-2.0944	1.5708	True	1	Angle [rad]
26	Position of Joint arm_rot_l	-1.5708	1.5708	True	1	Angle [rad]
27	Position of Joint elbow_flex_l	-0.3	2.618	True	1	Angle [rad]
28	Position of Joint pro_sup_l	-0.6	1.5708	True	1	Angle [rad]
29	Position of Joint wrist_flex_l	-1.22173	1.22173	True	1	Angle [rad]
30	Position of Joint wrist_dev_l	-0.436332	0.610865	True	1	Angle [rad]
31	Velocity of Joint pelvis_tx	-inf	inf	False	1	Velocity [m/s]
32	Velocity of Joint pelvis_tz	-inf	inf	False	1	Velocity [m/s]
33	Velocity of Joint pelvis_ty	-inf	inf	False	1	Velocity [m/s]
34	Velocity of Joint pelvis_tilt	-inf	inf	False	1	Angular Velocity [rad/s]
35	Velocity of Joint pelvis_list	-inf	inf	False	1	Angular Velocity [rad/s]
36	Velocity of Joint pelvis_rotation	-inf	inf	False	1	Angular Velocity [rad/s]
37	Velocity of Joint hip_flexion_r	-inf	inf	False	1	Angular Velocity [rad/s]
38	Velocity of Joint hip_adduction_r	-inf	inf	False	1	Angular Velocity [rad/s]
39	Velocity of Joint hip_rotation_r	-inf	inf	False	1	Angular Velocity [rad/s]
40	Velocity of Joint knee_angle_r	-inf	inf	False	1	Angular Velocity [rad/s]
41	Velocity of Joint ankle_angle_r	-inf	inf	False	1	Angular Velocity [rad/s]
42	Velocity of Joint hip_flexion_l	-inf	inf	False	1	Angular Velocity [rad/s]
43	Velocity of Joint hip_adduction_l	-inf	inf	False	1	Angular Velocity [rad/s]
44	Velocity of Joint hip_rotation_l	-inf	inf	False	1	Angular Velocity [rad/s]
45	Velocity of Joint knee_angle_l	-inf	inf	False	1	Angular Velocity [rad/s]
46	Velocity of Joint ankle_angle_l	-inf	inf	False	1	Angular Velocity [rad/s]
47	Velocity of Joint lumbar_extension	-inf	inf	False	1	Angular Velocity [rad/s]
48	Velocity of Joint lumbar_bending	-inf	inf	False	1	Angular Velocity [rad/s]
49	Velocity of Joint lumbar_rotation	-inf	inf	False	1	Angular Velocity [rad/s]
50	Velocity of Joint arm_flex_r	-inf	inf	True	1	Angular Velocity [rad/s]
51	Velocity of Joint arm_add_r	-inf	inf	True	1	Angular Velocity [rad/s]
52	Velocity of Joint arm_rot_r	-inf	inf	True	1	Angular Velocity [rad/s]
53	Velocity of Joint elbow_flex_r	-inf	inf	True	1	Angular Velocity [rad/s]
54	Velocity of Joint pro_sup_r	-inf	inf	True	1	Angular Velocity [rad/s]
55	Velocity of Joint wrist_flex_r	-inf	inf	True	1	Angular Velocity [rad/s]
56	Velocity of Joint wrist_dev_r	-inf	inf	True	1	Angular Velocity [rad/s]
57	Velocity of Joint arm_flex_l	-inf	inf	True	1	Angular Velocity [rad/s]
58	Velocity of Joint arm_add_l	-inf	inf	True	1	Angular Velocity [rad/s]
59	Velocity of Joint arm_rot_l	-inf	inf	True	1	Angular Velocity [rad/s]
60	Velocity of Joint elbow_flex_l	-inf	inf	True	1	Angular Velocity [rad/s]
61	Velocity of Joint pro_sup_l	-inf	inf	True	1	Angular Velocity [rad/s]
62	Velocity of Joint wrist_flex_l	-inf	inf	True	1	Angular Velocity [rad/s]
63	Velocity of Joint wrist_dev_l	-inf	inf	True	1	Angular Velocity [rad/s]
64	3D linear Forces between Back Right Foot and Floor	0.0	inf	True	3	Force [N]
67	3D linear Forces between Front Right Foot and Floor (If box feet is disabled)	0.0	inf	True	3	Force [N]
70	3D linear Forces between Back Left Foot and Floor	0.0	inf	True	3	Force [N]
73	3D linear Forces between Front Left Foot and Floor (If box feet is disabled)	0.0	inf	True	3	Force [N]

Action Space

The action space has the following properties by default (i.e., only actions with Disabled == False):

(min=-1, max=1, dim=13, dtype=float32)

The action range in LocoMuJoCo is always standardized, i.e. in [-1.0, 1.0]. The XML of the environment specifies for each actuator a gearing ratio, which is used to scale the the action to the actual control range of the actuator.

Some actions are disabled by default, but can be turned on. The detailed action space is:

Index	Name in XML	Control Min	Control Max	Disabled
0	mot_lumbar_ext	-1.0	1.0	False
1	mot_lumbar_bend	-1.0	1.0	False
2	mot_lumbar_rot	-1.0	1.0	False
3	mot_shoulder_flex_r	-1.0	1.0	True
4	mot_shoulder_add_r	-1.0	1.0	True
5	mot_shoulder_rot_r	-1.0	1.0	True
6	mot_elbow_flex_r	-1.0	1.0	True
7	mot_pro_sup_r	-1.0	1.0	True
8	mot_wrist_flex_r	-1.0	1.0	True
9	mot_wrist_dev_r	-1.0	1.0	True
10	mot_shoulder_flex_l	-1.0	1.0	True
11	mot_shoulder_add_l	-1.0	1.0	True
12	mot_shoulder_rot_l	-1.0	1.0	True
13	mot_elbow_flex_l	-1.0	1.0	True
14	mot_pro_sup_l	-1.0	1.0	True
15	mot_wrist_flex_l	-1.0	1.0	True
16	mot_wrist_dev_l	-1.0	1.0	True
17	mot_hip_flexion_r	-1.0	1.0	False
18	mot_hip_adduction_r	-1.0	1.0	False
19	mot_hip_rotation_r	-1.0	1.0	False
20	mot_knee_angle_r	-1.0	1.0	False
21	mot_ankle_angle_r	-1.0	1.0	False
22	mot_subtalar_angle_r	-1.0	1.0	True
23	mot_mtp_angle_r	-1.0	1.0	True
24	mot_hip_flexion_l	-1.0	1.0	False
25	mot_hip_adduction_l	-1.0	1.0	False
26	mot_hip_rotation_l	-1.0	1.0	False
27	mot_knee_angle_l	-1.0	1.0	False
28	mot_ankle_angle_l	-1.0	1.0	False
29	mot_subtalar_angle_l	-1.0	1.0	True
30	mot_mtp_angle_l	-1.0	1.0	True

Rewards

The default reward function is based on the distance between the current center of mass velocity and the desired velocity in the x-axis. The desired velocity is given by the dataset to imitate.

Class: loco_mujoco.utils.reward.TargetVelocityReward

Initial States

The initial state is sampled by default from the dataset to imitate.

Terminal States

The terminal state is reached when the humanoid falls, or rather starts falling. The condition to check if the robot is falling is based on the orientation of the robot, the height of the center of mass, and the orientation of the back joint. More details can be found in the _has_fallen method of the environment.

Methods

static generate(task='walk', dataset_type='real', **kwargs)

Returns a Humanoid environment and a dataset corresponding to the specified task.

Args:

env (class): Humanoid class, either HumanoidTorque or HumanoidMuscle. path (str): Path to the dataset. task (str): Main task to solve. Either “walk” or “run”. dataset_type (str): “real” or “perfect”. “real” uses real motion capture data as the reference trajectory. This data does not perfectly match the kinematics and dynamics of this environment, hence it is more challenging. “perfect” uses a perfect dataset. debug (bool): If True, the smaller test datasets are used for debugging purposes.

Returns:

An MDP of a Torque Humanoid.

valid_task_confs = <loco_mujoco.environments.base.ValidTaskConf object>

Torque Humanoid 4 Ages

class HumanoidTorque4Ages(**kwargs)

Bases: BaseHumanoid4Ages

MuJoCo simulation of 4 simplified humanoid models with one torque actuator per joint. At the beginning of each episode, one of the four humanoid models are randomly sampled and used to simulate a trajectory. The different humanoids should resemble an adult, a teenager (∼12 years), a child (∼5 years), and a toddler (∼1-2 years). This environment can be partially observable by using state masks to hide the humanoid type indicator from the policy.

Note

This environment is almost identical to the HumanoidTorque environment. The only differences are a small addition to the observation space, a scaled reward function, and a slightly different initial state distribution.

Observation Space

Additionally to the HumanoidTorque model, the HumanoidTorque4Ages model has an identifier to give information about the current scaling. The identifier is a binary number (bits). The length of the binary number is determined by the number of scalings available. By default, it is 4, which is why the length of the identifier is 2 (Two bits are needed to encode a decimal <=4). By adding the identifier, the environment is not partially observable by default. The additional observation space is:

Index	Description	Min	Max	Disabled	Dim	Units
77	Binary number identifying the humanoid scaling.	0	1	False	2	None

Rewards

The default reward function is based on the distance between the current center of mass velocity and the desired velocity in the x-axis. The desired velocity is given by the dataset to imitate. In comparison to the HumanoidTorque model, this reward used a scaled desired target velocity to adapt to smaller humanoids.

Class: loco_mujoco.utils.reward.MultiTargetVelocityReward

Initial States

At the beginning of each episode, a scaling factor is randomly sampled to choose the type of humanoid. Based on that, the initial state is sampled by from the respective humanoid dataset.

Methods

static generate(task='walk', mode='all', dataset_type='real', **kwargs)

Returns a HumanoidTorque4Ages environment corresponding to the specified task.

Args:

task (str):

Main task to solve. Either “walk” or “run”.

mode (str):

Mode of the environment. Either “all” (sample between all humanoid envs), “1” (smallest humanoid), “2” (second smallest humanoid), “3” (teenage humanoid), and “4” (adult humanoid).

dataset_type (str):

“real” or “perfect”. “real” uses real motion capture data as the reference trajectory. This data does not perfectly match the kinematics and dynamics of this environment, hence it is more challenging. “perfect” uses a perfect dataset.

Returns:

An MDP of a set of Torque Humanoid of different sizes.

valid_task_confs = <loco_mujoco.environments.base.ValidTaskConf object>