Tag Archives: moteus

Filtering encoder values in moteus

robots, , ,

TLDR: moteus can now filter the encoder, resulting in less audible noise. Use firmware version 2021-04-20 and ‘pip3 install moteus’ version 0.3.19, then re-calibrate to get the benefits.

Background

The moteus controller uses an absolute magnetic encoder to measure the position of the rotor. It uses this knowledge to accurately control the current through the three phases of a brushless motor so that the desired torque is produced, i.e. “field oriented control”. This works well, but has some downsides. One, is that magnetic encoders work by sensing the magnetic field produced by a “sensing magnet” that is somehow affixed to the rotor. This sensing process always introduces some noise, so that the sensed rotor position is never perfect.

Because of this, even if the rotor is perfectly stationary, moteus will constantly be tweaking the phase of the motor current to track the noise. This results in slightly increased power consumption, and more important to most, audible noise as the slight variations in control current can induce vibrations in the phase windings of the motor.

Fortunately, in most applications, the rotor is not actually capable of accelerating with the full bandwidth that the magnetic sensor is capable of sensing. Thus, we can filter it to remove high frequency noise without any loss of performance.

Approach

The filter method that moteus uses is an “all-digital phase locked loop“. Formulated in the traditional PLL terminology, this consists of three pieces: a numerically controlled oscillator, a phase detector, and a PI controller.

The “numerically controlled oscillator” is just a counter that tries to “guess” what the encoder is doing by propagating a value forward at an estimated velocity. For an ADPLL in this setup (where our encoder gives us an absolute phase measurement), the “phase detector” just subtracts the measured phase from the estimated phase. The PI controller determines how the estimated oscillator tracks disturbances in the actual encoder frequency.

In my post on automatic torque bandwidth selection for moteus, I derived the necessary equations for determining the torque bandwidth. Here, I’m just going to reference the excellent article by Neil Robertson, “Digital PLL’s” (part 1, and part 2), over at dsprelated.com. There, he derived the necessary proportional and derivative gains for a given bandwidth and damping ratio:

K_p = 2 \zeta \omega_{3db}

K_i = {\omega_{3db}}^ 2

Where \zeta is the “damping ratio“. For our purposes, we’ll use 1.0, which is known as “critically damped” and is often a good balance between stability and response time.

Experiments

To see this in action, I ran some experiments on a moteus devkit. I added back in a conditional ability to emit a few bytes of debugging information at the full control rate to the firmware, and used that to emit the encoder value as used by FOC at each control cycle. I made up a script, that can plot the power spectral density in the encoder measurements. This is basically the noise present at any given frequency. When run on an unmodified development kit with no encoder filtering, the result looks like:

Power spectral density of moteus AS5047P encoder in a devkit

This shows that there are peaks in the noise at around 700Hz and 2kHz and after that the noise drops off rather rapidly. Then if we enable filtering at a few different bandwidths, you can see how the plot changes:

Power spectral density of AS5047P with various filter bandwidths configured

The filter bandwidths here varied from 5000 rps (~800Hz) down to 100rps (~15Hz). The filtered noise profiles follow what one would expect for each of the filters.

Audible noise

To measure the combined effect of this and torque bandwidth on audible noise, I switched to a controller attached to an 8108 motor, as I’ve seen those tend to demonstrate more audible noise. The motor had a constant back and forth motion at 0.5Hz for 60s and the result was captured with a studio mic set about 1cm away from the motor. In each test, I ran moteus_tool --calibrate and selected only a torque bandwidth, letting it pick an appropriate encoder bandwidth. For the 100Hz torque bandwidth and above tests, the same position PID values were used, although they needed to be tweaked for stability at the lower bandwidths. I similarly plotted the power spectral density for several filtering values and for silence. The total signal strength measured in negative dB is noted in the label:

It is not shown here, but the unfiltered noise is about the same as the 400Hz one. Thus, the maximum improvement is around 6dB of audible noise. Filtering at 100Hz gives most of that benefit for this motor, with slight improvements beyond that. Most of the audible energy is in the spectrum below 1kHz and there are several frequency bands where the encoder does not appear to be the dominant source of audible noise, as all options are equivalent in those.

For comparison purposes, the ODrive firmware defaults to filtering both the torque and encoder at around 160Hz (1000rps).

moteus_tool integration

As of version 0.3.19, moteus_tool has support for configuring the encoder bandwidth during calibration. By default, it will select a value appropriate for the selected torque bandwidth, or you can specify it manually with --encoder-bw-hz.

moteus direction configuration

robots,

Since the first public release, moteus has always calibrated motors so that a positive command is equivalent to a fixed sequencing of the phase wires. That means that depending upon which order you solder the phase wires, a positive commanded velocity will result in the motor spinning either clockwise or counterclockwise.

As it turns out, since moteus has an absolute encoder that is immune to such vagarities, it is much more convenient to normalize the direction of rotation around the encoder rather than the phase wires. As of release 2021-04-26, and moteus_tool 0.3.22, that is exactly what moteus does.

Now by default, moteus_tool will attempt to calibrate the motor such that a positive command results in a positive encoder value (this will be clockwise rotation when looking at the front of the motor / back of the moteus board). If the firmware is too old, it may not be able to do this, in which case it will emit an error.

You can also, use the new ‘--cal-invert' option, which will have moteus calibrate the motor such that a positive command results in the encoder moving in the negative direction. For some existing deployed systems, you must use this option if you want the calibrated motor to have the same sense of direction as before (even with old firmware). As before, if this is not possible because the firmware is too old to support the necessary configuration, an error is emitted.

Auxiliary encoders for moteus

robots, , , ,

The moteus controller uses an absolute magnetic encoder to sense the position of the rotor in order to conduct field oriented control of the motor. In many applications, this sensing is also sufficient to measure the output as well, particularly in direct drive applications. However, if the controller is driving the output through a gear reduction, multiple turns of the input are necessary to make one turn on the output. At power on, this results in an ambiguity, where the controller doesn’t know where the output is.

There are a couple of possible solutions to this, one is to do like the quad A1 does, and have a “known turn on position”. Another would be to have a rigid end stop and use a homing procedure on startup. Yet another would be to have a non-backdrivable mechanism and remember in the host application how many revolutions had been taken.

Auxiliary encoders

What I’m going to cover here is yet another solution to this problem, an auxiliary encoder. In this approach, a second absolute encoder is used to measure the position at the output directly, thus directly resolving all ambiguity. All of the production moteus controllers have had a, to date unused, connector named ABS which has pins intended for I2C on it. As of revision 2021-04-09, moteus can now use these pins to read the position from an AS5048B absolute magnetic encoder.

After reading, it uses the values for two purposes. First, it reports the measured value out over both the diagnostic and register interface, so that host applications can use it. It also can be optionally used to initialize the value of “unwrapped_position” at startup.

Setting it up

Getting an auxiliary encoder working is pretty easy. First, you need to wire something up. If you don’t already have hardware, you can use these cables from amazon, and this breakout board from digikey. Here’s a photo of those connected up with with a .1″ connector in between.

The pinout on the moteus board is described in the reference manual:

  • Pin 1: 3.3V (closest to ABS label)
  • Pin 2: SCL
  • Pin 3: SDA
  • Pin 4: GND

Then you connect it to the moteus (while off) and install the breakout board facing a magnet. Here, I made a simple 3d printed belt reducer:

Now we can go into tview and configure things. First, we use the config abs_port.mode from the reference manual, and set it to the value for an AS5048B (1). Then we will configure an offset using abs_port.position_offset, and finally set the position to be set from this encoder on startup with servo.rezero_from_abs.

And that’s all that is necessary!

Using it

At this point, the current value of the encoder can be read at in tview at 'abs_port.position' or register 0x006 as documented in the reference manual!

Here’s a video showing a bit more detail:

New available limits in moteus firmware

robots,

Some of the new features in the moteus 2021-04-09 firmware release are new limits that can make the overall system more robust to faults or environmental conditions.

Velocity limiting

config: servo.max_velocity

Like the position limits, there now is a configurable velocity limit. If the motor is moving faster than this limit in either direction, then the applied torque will be limited, eventually to a value of 0. This can be used to reduce the likelihood of runaway behavior in systems where high speeds are not expected.

moteus development kits come with this value set to 10 revolutions per second. It can be easily changed in tview.

Power limiting

config: servo.max_power_W

This limit throttles the available PWM when the overall power applied to the motor exceeds this configured value. By default it is set to 450W, slightly less than the moteus’s rated power of 500W. This limit is intended to reduce the likelihood of damaging the controller through an over-power situation.

Auto-tuning current control loops

robots, , ,

Since the moteus controller was first released, it has implemented a two-stage controller. The outer loop is a combined position/velocity/torque PID controller, which takes as an input a position trajectory, and outputs a desired torque. The inner loops accepts this torque, and uses a PI controller to generate the Q phase voltage necessary to achieve that torque.

Until now, the constants for that PI controller were left as an exercise for the reader. i.e. there were some semi-sensible defaults, but the end-user ultimately had to manually select those constants to achieve a given torque bandwidth. That isn’t too much of a problem for a sophisticated user, but for the rest of us, it is hard to know how to go from a desired torque bandwidth to reasonable PI gains.

Now, I’ve finally decided to tackle this problem! There are two phases, the first being how to measure the inductance of a given motor, which is last major motor parameter to be automatically measured. The second is how to generate a controller using the resistance and inductance which has a desired bandwidth.

Measuring inductance

An inductor is an energy storage device where the rate of change of current is directly proportional to the voltage across its terminals. Most (probably all?) electrical motors act as inductors in various forms. The hobby outrunners typically driven by moteus controllers, surface permanent synchronous magnet motors, can be treated as two equal valued inductors across the imaginary D and Q axes.

Given an ideal inductor, an easy way to measure that inductance is to apply a constant voltage and see how fast the current rises. That’s roughly the approach I took with moteus, but instead of a fixed voltage, it applies a square wave voltage centered about 0 across the D axis. That way the motor doesn’t move, the net current is 0 over time, and the average rate of change can be integrated over many cycles for much higher accuracy.

This “square wave” mode is implemented as a new control mode in moteus, primarily intended to be invoked by moteus_tool during the calibration process. You can specify the half-period of the square wave in control cycles, and the voltage to apply across the D axis. Here’s a plot of the current, as reported by a moteus controller over its debug DAC connection during a test on an mj5208:

This shows that yes, the current is roughly triangular for the input square wave.

Designing a current controller

Given this newly measured inductance, and the phase resistance that the calibration process already measured, we have sufficient information to design a PI controller which achieves a desired bandwidth response.

For a PI controller, a simple second order approximation of the closed loop transfer function can be written as:

I(s)=\frac{K_p s + K_i}{L s^2 + (R+K_p) s + K_i}

Where L is the D/Q phase inductance, R is the phase resistance, and Kp and Ki are the PI gains. Taking that, if we constrain:

K_i = K_p \frac{R}{L}

We can then substitute this into the original transform, which results in a first order system:

I(s)=\frac{K_p}{L} \frac{1}{s + \frac{K_p}{L}}

This system has a 3db frequency at the sole pole:

w_{3db} = \frac{K_p}{L}

Which then allows us to fully determine all the remaining parameters:

K_p = w_{3db} L
K_i = w_{3db} R

Finally, when we double check our results, the “rise time” for a first order system (roughly the time it takes to go from 10% to 90%) can be described as:

t_{rise} = \frac{0.35}{BW_{hz}}

A motor like the mj5208 can be approximated as a phase resistance of 0.04 ohm, with a D/Q phase inductance of 25e-6 H. For a 3dB bandwidth of 1000 radians per second (~160Hz), the gains measured in radians per second would be:

K_p=0.025
K_i = 40.0

To set these on the controller, we would use the following parameters:

servo.pid_dq.kp=0.025
servo.pid_dq.ki=40.0

Empirical testing

To make sure these make sense, I used this process for a few different bandwidths on a few different motors. For each, I used an oscilloscope to measure the rise time of the current waveform in response to a step input of a relatively small 4A command. The shape of that curve lets us be pretty confident that the damping ratio is correct, and the total rise time of the curve gives a good measure of the overall bandwidth that was achieved.

Across the motors I tested, the resistance varied from 35-65 milliohm, with inductances varying from 9uH to 33uH. The grid shows that the bandwidth worked out pretty close to the target in all those cases.

moteus_tool integration

Now that the math is out of the way, you don’t need to worry about it at all, because as of version 0.3.17 of moteus_tool (with version 2021-04-09 of the firmware), this whole process is automated. When you run the calibration step, you can just specify the desired 3db bandwidth on the command line of moteus_tool and it will calculate and set the parameters for you. By default it will select 100Hz bandwidth.

python3 -m moteus.moteus_tool -t 1 --calibrate --cal-bw-hz 100

Video

If you want to see this content explained more verbosely, here’s a video that covers most of it!

Unsuccessful CAN-FD communication between CANBed-FD and moteus

robots, , ,

On the mjbots discord, people are often looking for the cheapest possible way to command and monitor a moteus controller. One possible solution that comes up over and over again is the CANBed-FD board, as sold by Seeed Studio (and others). I decided to get one of these in house to see if I could make it work:

The first thing I figured out was that the DB-9 connector used a non-standard pinout for CAN_L and CAN_H. I just switched to the terminal block connections instead of the DB-9 to get around that. For the software, I decided to use the acan2517FD Arduino library, as it was quite a bit more robust and featureful than the one provided by Longan Labs.

With this in hand, I was able to get the Arduino to send properly formatted CAN-FD frames that moteus would receive. However, as it often does, the problem came when receiving. This Longan Labs board uses a 20MHz clock for the MCP2518FD onboard, which means that it is only *just* able to do the 5MHz CAN-FD clock rate that moteus requires, and there is no margin for actually being able to synchronize to a slightly skewed bitrate using the sync jump width. Because of this, I was not able to get the device to receive data from a moteus controller except by nudging the bitrate of the moteus controller around by fractional percents manually, and then it only worked for a short time before temperature, or humidity, or whatever, drifteed it out of range.

If someone finds an Arduino board that uses a MCP2518FD with a 40MHz clock (or a 4MHz clock that uses the built in PLL), let me know, as I expect that should be able to work fine. That is the configuration used by the Seeed Raspberry Pi CAN-FD adapter, which has 2 MCP2518FDs with a 40MHz clock and it has been demonstrated to work with bit timings listed in the moteus reference documentation.

mjbots power_dist r4.3b

robots, ,

I’d like to introduce the newest mjbots product, an updated revision of the power_dist, 4.3b available at mjbots.com today!

This version has a number of improvements over the previously released r3.1:

r4.3br3.1
Voltage Range10-44V8-34V
Maximum load capacitance4,000 uF400 uF
Quiescent Current300uA5mA
Current (Continuous / Peak)45A / 80Aunrated / 100A
Energy MonitoringYESNO
Switch ModeHigh SideLow Side
Dimensions50x80mm45x70mm
Price$139$79

The only real downsides are that is more expensive and slightly larger.

If you want to read up more on the motivation and design process for this board, you can see that in (part 1, part 2, part 3, part 4).

Hot swap controllers (next get power_dist part 3)

robots,

This is one of a series covering the new mjbots power_dist board. See part 1 and part 2 for more context.

As mentioned previously, hot swap controllers are primarily used to allow a card to be inserted live into a server backplane, while minimizing disruption to the primary power bus while doing so. Additionally, they often implement protection features like over-current and short-circuit protection, and some support energy monitoring.

Typical topology

A typical hot-swap topology looks like:

Here, the high side FET is used for two purposes. During the “pre-charge” phase, the FET is operated in the linear regime, with a large voltage across it until the load capacitance is fully charged, at which point the FET reaches its “fully on” state and operates with its stated Rds on resistance. Notably, this means that the entire pre-charging energy is dissipated in those FETs, as opposed to the r3.1 design where a power resistor serves that purpose.

To accomplish the protection functionality, the current shunt (R1) is used to measure total current moving through the primary FET. Depending upon the particular chip, this could include over-current monitoring, where the FET is brought back into the linear operating region to limit the current, or short-circuit protection, where the FET is immediately turned off. Additionally, the resistor divider (R2/R3) can be used to program an undervoltage threshold.

Design constraints

The biggest challenge that a design faces with any hot swap controller is selection of the primary FET. Because it has to dissipate the entire pre-charge energy in a short time window, the device becomes very stressed. This limits the total capacitance that can be charged, the maximum voltage, and somewhat non-intuitively, the current that can be drawn during this startup window. For a design like the power_dist, where there is no “power good” signal distributed to downstream loads, they can draw what are effectively constant-power loads soon after the bus voltage reaches a valid intermediate state. Given that this can occur when there is still a large voltage across the high side FET, it can add a lot of energy dissipation.

To make this even harder, the design constraints are such that during the critical pre-charge window, FETs cannot be naively be placed in parallel. When operating in the linear regime, minute differences in device characteristics and temperature can cause drastic load imbalances. Thus controller design equations only allow parallel FETs for the purposes of increasing steady state on current, not increasing power or capacitance during the pre-charge window.

For a given FET, it will have a “safe operating area” plot like the below (this from the PSMN3R7-100BSE used in the soon-to-be-released power_dist r4):

This shows how much energy can be safety dissipated over different time windows. When designing the pre-charge system, it is then a balancing act of getting things to charge as quickly as possible, while not violating the SOA. Because of the details, going faster or slower can be problematic. Faster can be an issue because the energy may violate the short term peak energy, slower may be the culprit because the initial load current spends a longer time moving through the FET when it has a large voltage across it.

Next steps

In the next post, I’ll cover my various iterations, and where they fell short.

Next-gen power_dist (part 2)

robots,

Last time I covered the limitations of the power_dist r3.1, here I’ll cover some iterations of the design process.

My initial design goals for this version are based largely around improving the major limitations identified before:

  • Positive side switching: By switching the positive rail, a whole class of use failures is removed, as most people expect ground to be common throughout a system.
  • Increased voltage range: moteus r4.5 and the pi3hat both support 44V, so any new power_dist board should support at least that.
  • Lower quiescent current: Ideally, the quiescent current would be measured in microamps, or at least at a level that it does not confuse BMS systems.
  • Energy monitoring: Often in the development of the quad A1, I wanted to have a system level power and energy monitoring solution so as to identify the energy cost of various maneuvers and gaits. Tracking that at the power_dist level seems like a logical place.
  • Wider load envelope: The 3.1 version had a relatively limited maximum downstream capacitance and turn-on current draw. It was enough to power on 12 moteus controllers and a small computer, but not much else.

To achieve these goals, I decided to try using what is known as a “hot swap controller”. These are integrated circuits that are intended for use in cards that plug into server backplanes. Given that any given card could potentially have a large decoupling capacitance, inserting it live into a backplane could cause arcing, and high currents that cause the overall bus voltage to drop outside of tolerable limits.

While such controllers are intended to be used at the point of load, they are largely applicable in this centralized system too, where the “hot plugging” is of a battery instead of an individual load.

And here is where my design story becomes “complicated”. Because both I didn’t fully work through the consequences of each design approach, and because I was not familiar with the intrinsic limitations of hot swap controllers, I ended up taking a route that involved 4 different prototypes before I arrived at one that was mostly acceptable. Rather than go down that route linearly, in the next post I’ll instead distill what I learned about hot swap controllers and design before describing the final solution.