I documented the first test fixture I built for moteus some time ago. As the shipment volumes have gone up, the fixture became something of a limitation, and also was a little problematic in a few ways.

The old “state of the art”

First, it relied on attaching 3 connectors by hand for each test, which was a decent fraction of the cycle time. Second, the pogo pins it used were non-replaceable, and also connected only to the debug phase wire test vias, which were tiny. They wore out relatively quickly, and replacing them required building a whole new board. Finally, since the pogo pins were PCB mounted, a PCB needed to be printed for any change in the pin locations or which pins to probe.

New fixture

Enter the new fixture:

This one uses modular test probe receptacles with replaceable tips, so that the tips can just be swapped out as they wear. These don’t appear to be available in an inexpensive manner in the US, but aliexpress has many options with 5 or 6 probe types in many sizes.

Second, the receptacles are all aligned with a 3D printed baseplate and platen to align them before contacting the PCB. This works alright down to the 1.5mm spacing test points on moteus, although that pitch is pushing what can be achieved with a 3D printed part.

Third, power, CAN, and the debug serial connections are now all probed from the bottom, so only one connector, the SWD programming port needs to be connected by hand.

Finally, the whole structure, including the top clamp is 3D printed now, which makes it potentially possible to do top probing and more easily adjust the dimensions.

With this new fixture, my cycle time for a test is around 60s. At that point, the time spent in the test program is about the same as the time it takes to unpack and package up the boards, so it isn’t really the limiting factor any more.

For context, see part 1, part 2, or part 3.

r4.0

My first attempt at an r4 design was based on the TI LM5066 hot swap controller. It is one of the more full featured controllers, since it supports built in energy monitoring over a SPI bus with no additional components. This first iteration was actually surprisingly close to being workable. There were two factors that it performed poorly on, quiescent current and energy monitoring. The quiescent current was similar to the r3.1 version. Energy monitoring was present, but at the full scale range necessary for power_dist, it was almost unusably inaccurate. With a design set for a peak of 100A, and also the lower 25mV current sense range, the current noise was measured in multiple amps.

I definitely wanted usable energy monitoring, and wasn’t quite ready to sacrifice the quiescent current, so decided to try again.

r4.1

This version was based on the Analog LTC4380, nominally a “surge stopper” rather than a hot swap controller. I was attracted to it because it seemed like it would offer a similar pre-charge function, but with a much reduced quiescent current. I implemented energy monitoring using a separate Analog LT1787 high side current sense monitor which was fed into the op-amps on the STM32G474.

This version had multiple new and different problems. For one, the “fault timer” on the LT4380 is active during the “pre-charge” window. It turned out to be impossible to select a pre-charge speed that would handle a reasonable capacitance while simultaneously not triggering the fault timer.

Secondly, I discovered that the LT1787 has a relatively high output impedance, such that significant error was introduced by feeding it directly into an inverting op-amp with gain on the STM32G4. The G4 is relatively configurable, but there was no way with the existing PCB to first run it through a buffer op-amp to improve the impedance before moving to a gain stage (and only then on to the onboard ADC).

Finally, this version had the first incarnation of a “latching” circuit that would allow the processor to shut off power to the entire 3.3V bus when desired, but flipping the primary power switch would turn it back on. Thus the processor could decide how long to run after the primary power switch was turned off. Unfortunately, this circuit had several problems in this iteration that needed resolving.

r4.2

This version used the TI TPS2490 hot swap controller along with the same LTC1787 high side current sense amplifier. It re-arranged the current sense input so that the STM32G4 could first buffer, then amplify the signal before handing it off to the ADC. The biggest problem here was once again the circuit that attempted to let the STM32G4 shut itself off to lower quiescent current. It ended up being unworkable, and also rendered the TPS2490’s under-voltage protection non-functional.

r4.3

This version is basically the one that went into production. It still uses the TPS2490 and LTC1787, but now has a simpler latching mechanism, and undervoltage protection that actually works.

r4.3b

The only changes here are silk-screen modifications!

Next up

In the next, and final post, I’ll introduce r4.3b and its specifications!

For those who have been annoyed by the misplaced I2C connector on the r4.3 moteus model, I’ve finally updated the 2D and 3D mechanical drawings for the moteus controller in github:

https://github.com/mjbots/moteus/tree/main/hw/controller/r4.5

And in the same vein, we received a great user contribution from Rüdiger Koch-Kukies in the form of a solid model of the mj5208 motor!

Sometimes you find bugs, and sometimes the bugs find you! While getting ready to release some updated qdd100 beta servos, I was investigating speed control in the moteus firmware and discovered an unwelcome surprise!

Voltage feedforward

To improve dynamic response under high accelerations, moteus uses a voltage feedforward term as part of its control loop. When trying to command a specific current, it normally applies a PI controller to determine the resulting D and Q phase voltages to apply. The feedforward component of this looks at the phase resistance and the current rotor velocity, and determines a zeroth order estimate of what voltage would be required to achieve that current under a no-load condition, only using the PI controller to determine the difference.

This technique has been in use since my earliest direct drive leg jumping experiments. However, at some point before I added support for gearing reduction in the controller, there was a lot of confusion in the code base around the sign of the torques vs. the sign of the encoder. When that was sorted out, I missed the fact that the resolution ended up leaving the velocity component of the feedforward with a *negative* value. This meant that instead of helping the PI controller, it made the PI controller have to work twice as hard!

The easy, and not-so-easy fix

Fixing this for new controllers involved changing just a single character change from ‘-‘ to ‘+’. However, there is a configuration value which applies an overall scaling to this feedforward term, and has been 1.0 for all released moteus controllers. With the correct sign, a 1.0 scaling factor can result in unstable performance at high velocities.

Knowing this probably won’t be the only complicated upgrade procedure for the moteus controller, I decided to invest in a little infrastructure to make it easier in the future. This is built on two components:

1. The “firmware version” was always stored in the firmware image and reportable over CAN. Until now, it has always been 0x00000100. I’ve now set this up so that the version is stored as a top level symbol in the firmware .elf file that automated tools can inspect.
2. moteus_tool now checks the existing firmware version, and the new firmware version. When crossing given firmware versions, it can apply programmatic rules to update the configuration to maintain as close as possible semantic values. To make that robust, it refuses to flash firmware versions newer than it is aware of.

Version 2021-01-14

Thus, the new release, 2021-01-14 which has two improvements / bug fixes:

1. The aforementioned feedforward sign error.
2. Temperature values are now minimally filtered before being used for derating or faults.

If you want to upgrade, be sure to use a moteus_tool version 0.3.7 or newer from pypi. Or, if not, make sure that servo.feedforward_scale is set to no more than 0.5 when you’re done!

I took a lot more footage than ended up in the annual update video. Here’s a short with footage from Killian Court on the Charles River:

When I first posted the moteus controller up for sale, the specifications I listed were based on the design characteristics and the testing I had conducted up until that point. Specifically, the peak phase current was just the maximum that I had verified was safe to operate with.

In the interim, I’ve done a fair amount more testing and have concluded that the controller can safely drive higher phase currents than initially posted. For now, I’m increasing the peak phase current to 100A both on the specification sheet and in the default firmware configuration of all newly shipped boards. If you already have a moteus controller and want to take advantage of the higher phase current, chat us up on the mjbots discord and we’ll show you how.