New compilation commands for moteus

To stay on top of bazel development, and to prepare for some future improvements, I’ve gone ahead and upgraded the moteus firmware build system, rules_mbed to use the new bazel “platforms” toolchain resolution mechanism.

Previously, rules_mbed used the “crosstool_top” bazel mechanism for toolchain configuration. This allowed a single package to contribute a set of C++ toolchains which would be selected based on CPU and compiler. One of the downsides from the rules_mbed perspective, is that it made it difficult to make a build that included both mbed targets and host targets (or anything else non-mbed). rules_mbed worked around this by including a functioning clang host toolchain within it.

With the new toolchain resolution support, at the command line you specify to bazel what “platform” you want to be the final target. The updated rules_mbed specifies a “platform” for each of the STM32 processors that are supported. So for instance you could use:

bazel build --platforms=@rules_mbed//:stm32g4 //:target

It then uses that platform to find a toolchain with a compatible operating system and CPU, which for rules_mbed is the arm-gcc compiler for the correct chip.

Because of this, the command lines necessary to compile the moteus firmware and host side tools have changed. Rather than expose the raw bazel options, it now just uses a bazel config to abstract away whatever mechanisms are required. So, the two necessary new command lines are:

tools/bazel test --config=target //:target  # build firmware
tools/bazel test --config=host //:host # build host tools

Soon, we’ll use this capability to add some useful new features to the moteus tools, such as support for non-linux operating systems.

New leg cable management

Now that the quad A1 has been running faster, it has started “running” through its ad-hoc cable management too. After replacing a harness for the nth time, I decided to actually design something rather than just keep re-building over and over again.

My current best effort uses semi-flexible nylon split conduit, captured in 3d printed forms at each joint. Inside that conduit is basically the same harness I had before, with the cables selected to be more robust to repetitive motion. The nylon conduit is only semi-flexible, so it enforces a relatively large minimum bending radius on the wires within, while still sticking to the black quad A1 color motif.

To verify this would be at least a minimal step up, I ran some endurance testing with around 100,000 cycles of the leg moving. With no degradation of the cables in that window, I’m calling this good enough for this iteration of improvement.

Stable gait sequencing

In the last post, I described the newer gait engine which takes a desired command and produces a set of gait parameters. At that point, the gait engine needs to implement those gait parameters in a way that is stable with respect to disturbances and keeps the two legs properly out of phase with one another.

The gait variables that the gait selection procedure emits are as follows, each “leg” is actually a pair of legs.

  • swing time: This is the amount of time between when a leg lifts off the ground, and when it lands again.
  • one leg time: This is the amount of time spent with only a single leg on the ground.
  • two leg time: The amount of time with both legs on the ground.
  • flight time: The amount of time with no legs on the ground.

I’ve chosen to implement this for now with two controllable variables, when to pick up the next leg to begin its swing phase and how far out to place the foot when landing. When in flight, the swing time is fixed to be exactly what the gait engine selected. When on the ground, the leg position and velocity are fixed in the world frame to match the current command. This results in the other three parameters, one leg, two leg, and flight time, being approximated as best as possible.

When to lift

When there is no flight time, then the heuristic for when to lift the leg is identical to that described in “Balancing gait in 2D“. The trailing leg is lifted when the opposing leg is half of a swing time away from crossing the balance point.

When there is a flight time present, we have to add a corrective term to that heuristic, otherwise the two legs can lose their phase synchronization. If the correct out of phase relationship is being held, then the leg will be lifted when the alternate leg is in flight and has the “flight time” remaining in its swing cycle. The correction for that leg is calculated as a difference in time around that point such that the correction is positive before that point and negative after scaling in a linear manner. This has the desired property that the two legs are kept out of phase, without enforcing a strict leg cycle time.

Where to place

Once again, when there is no flight time, the placement location is the same as before. When there is a flight time, then the placement location is even simpler. Now it is just a position forward of the balance point by one half of the “one leg time”. This results in the machine spending roughly half of its time on each side of the balance point.

Further work

I’ve got one further enhancement to this technique to describe, at which point I think I’ll have basically exhausted its promise. Further advances will likely have to come from using an optimization based controller rather than a heuristic driven approach.

Higher speed gait formulation

As hinted in my earlier video I’ve been working towards some higher speed gaits with the quad A1. To accomplish that, I had to restructure the gait sequencing logic to permit changing cycle times and allow flight phases.

For now, I’ve tentatively broken down the trot gait into 5 regimes, based on how fast the machine is moving:

  1. At the slowest speeds, the flight legs swing through a step in the configured maximum flight time. The interval between flight times is fixed at a configured maximum. Here the speed is determined by how far the flight legs move.
  2. Once the flight legs are moving through their maximum allowed distance, then the amount of time spent with both legs on the ground is reduced in order to increase speed.
  3. At the point when both legs are not on the ground at the same time, then there begins to be a flight phase. Increasing the length of the flight phase increases the speed.
  4. When the flight phase reaches a configured maximum, then the swing time is decreased until it reaches a configured minimum.
  5. When the swing time is at a configured minimum, the flight time is at a configured maximum, and the legs are moving through their maximum range, then the machine is moving at its maximum speed.

Depending upon the current commanded rotation rate and translation velocity, the distance available for the legs to travel through may change. This uses the same mechanism from the step selection technique to determine the maximum distance at each update cycle, then selects which of the above regimes is active based on the commanded speed.

For a given maximum distance the legs can move through, that results in key gait parameters changing with speed as in the plot below:

Next up I’ll cover the heuristics used to implement any given set of gait parameters in a stable manner.

Unlimited rotations for moteus

The moteus controller has always supported multiple turns when counting positions. It has a one-revolution magnetic encoder built in, but after turn on, it keeps track of how many turns have occurred. However, if you’ve followed previous moteus tutorials, you have probably noticed a persistent caveat that for accurate control, the position of the output shaft needs to stay within a hundred revolutions of 0.0 or so. Now, I’ll describe why that was, and what I’ve done to remove the limitation, allowing unlimited rotations!

Background

The moteus controller uses a somewhat unique integrated position / velocity / torque control loop. This formulation gives a couple of advantages: First, there is no bandwidth loss due to having a cascaded position and velocity controller. Second, when driven by a higher level controller, it can seamlessly switch between position, velocity, and torque control, or any combination of them without having to manage mode transitions.

The command consists of the following values, all as 32 bit floating point values (optionally upscaled from integer values using the register protocol).

  • Desired Position
  • Desired Velocity
  • Feedforward Torque
  • kp scale
  • kd scale
  • Maximum Torque

The control loop measures two quantities as input, the “current position” and the “current velocity”. The position is measured as a 32 bit signed integer, where one revolution of the magnetic encoder equals 65536 counts. The velocity is numerically differentiated across the most recent 6.4ms of movement.

There are two internal state variables as well: One is the “target position”. This captures the most recent position command, and is advanced by the velocity command at the full control rate. The other is the integrative term of the PID controller. Both of these are stored as 32 bit floating point values.

The problem

This structure poses a few inherent limitations. One, being that as the control position is sent as a floating point value, the resolution available for positioning decreases as you get further from 0. That probably isn’t a big limitation, as there aren’t many applications where you want to have both absolute positions and also unlimited revolutions.

The bigger limitation is in the “target position” internal state variable. It needs to be updated to take into account the current velocity command at every control cycle, or 1/40000 of a second. For a commanded speed of 0.01 revolutions per second, this incremental update is only 2.5e-7 of a revolution. Given that 32 bit floating point values only have roughly 7 decimal digits of mantissa available to them, you don’t have to get far beyond 0 before an update that small doesn’t even change the value at all.

The command format also has an option, such that if the command position is set to a floating point NaN value, it will “capture” the current position. This can be used to command velocity-only control with an implicit integrative term, or when combined with a stop position to move to a target at a fixed velocity. However, since “capturing” stores the value as a floating point value, significant precision can be lost. This was only a problem at larger position values, but at the maximum position before wraparound, the available capture resolution was measured in multiple degrees.

The resolution

The resolution was relatively straightforward. Instead of storing the “target position” as a floating point value, it is now stored as a 64 bit integer measured in 1/(2**32) of a magnetic encoder revolution. This gives sufficient precision to represent velocities as small as 0.0001Hz (0.036 dps) uniformly at all positions, while still having more absolute range than the measured current position value. The final PID controller is then expressed relative to the target position. This lets it still operate in floating point coordinates, but with no worry about large artifacts due to a position offset.

The only other implementation hurdle was making it run fast enough. Largely that revolved around ensuring there was never a need to convert between 64 bit integers and floating point values, which is relatively slow on the STM32G4.

The result

With this fix in place, it is possible to operate the controller safely at high velocities for arbitrary periods of time. Even when the “current position” value wraps around from positive to negative! Also, low speed control works just as well at any position offset. When operating in those “continuous rotation” applications, the user should just be careful about if the “desired position” field of the command should be set. Largely, it should be left as NaN for when used in continuous rotation applications.

Here’s a video showing high speed wraparound and low speed at arbitrary offsets.

New “stay within” control mode for moteus

At the request of @nichols in discord, I’ve recently implemented a new control mode in the moteus controller, “stay within”. In this mode, as long as the controller is inside the currently commanded bounds, only a feedforward torque is commanded. When either of the optional lower or upper bound is violated, the normal PID controller is used to force the position back to the bound.

Here’s a quick video demo:

Note that this could have been roughly accomplished in a couple of ways by a higher level controller — either by monitoring the position and commanding zero kp/kd scales when inside the boundary, or just solely commanding feedforward torques based on position sensing. However, this approach lets the control run at the full 40kHz of the moteus controller, which results in much smoother operation at the boundary condition.

Optimizing moteus FET drive strength

The moteus controller uses a DRV8323 smart driver IC to drive the power MOSFETs as well as provide various safety functions. One of the capabilities it has which has so far been unexplored in moteus is its ability to control the drive strength and dead time through software configuration.

In a switching power supply or switching motor inverter, MOSFETs are arranged in a half bridge configuration. Depending upon the type of converter, one or more half bridges are used (3 phase inverters like moteus use 3 of them). Each “half bridge” has two MOSFETs, one connected between positive power and the output terminal, and the other connected between the output terminal and ground.

Power MOSFETs typically have relatively large gate capacitance, so to change their state quickly requires a lot of current. Additionally, you never want both MOSFETs conducting at the same time, otherwise current would flow straight from the supply to ground, called “shoot through”. Thus the driver has a configurable “dead time” which enforces that both are off for at least that long when switching (currents flow through the body diodes of the MOSFETs during this state).

Optimization

Selecting these parameters is a balancing act. If the drive current is too low, the MOSFETs take a long time to turn on and off, which means they spend more time in a high resistance state. At some point however, higher drive currents don’t make the MOSFETs switch any faster, and just burn more power in the driver without any benefit. Similarly for the dead time, if it is too low this will result in shoot through, and if too high, it will result in current flowing through the body diodes for longer, which is much less efficient.

Until now, I hadn’t done any real optimization of these parameters, aside from ensuring the system was functional within a safety margin. In advance of some other to be announced developments, I decided to take another look.

To make a test, I set up a moteus controller with a test motor, but set up so that there was no thermal connection between the motor and the controller, and that the controller was not heatsinked at all. That would allow me to more easily determine how much heat was coming from the controller itself. Then, for various supply voltages, I commanded a fixed D phase current with just enough Q phase voltage so that the motor gently spun around. This ensured that all 3 of the half bridges were used equally. Then, I waited until things had reached thermal equilibrium, and used my DIY thermal board inspector, to measure the temperature of the motor windings, FETs, and the DRV8323.

With that test methodology in hand, I was able to search and locate the optimal drive strength, and discovered that I can use the smallest available dead time with no problems.

MeasurementOld SettingsNew settings
DRV8323 @ 24V / 8A phase current73C69C
DRV8323 @ 32V / 8A phase current86C78C
FETs @ 24V / 8A phase current64C56C
FETS @ 32V / 8A phase current74C61C
Change in thermal equilibrium with no heatsinking

So, a nice win, especially at higher input voltages. The updated settings are in git master now, and will soon be in a new release.