Someone contacted me not too long ago who wanted to use the moteus controller, but wasn’t sure if it would be able to hit their target mechanical velocity of 6000rpm. I honestly wasn’t either, so I tested it. After a quick firmware fix, the devkit motor when run at 34V seems to be able to do it no problem.
It should be noted that the current firmware assumes you are within a thousand or so revolutions of 0. You can exceed that pretty quickly running at 120 revolutions per second!
One of my goals with mjbots is to make building dynamic robots more accessible to researchers and enthusiasts everywhere. To make that more of a reality, I’m lowering the prices in a big way on the foundational components of brushless robotic systems, the moteus controller and qdd100 servo.
As I am working to improve the gaits of the mjbots quad A1, one aspect I’ve wanted to tackle for a long time is improving the compliance characteristics of the whole robot. Here’s a small step in that direction.
Existing compliance strategy
The quad A1 uses qdd100 servos for each of its joints. The “qdd” in qdd100 stands for “quasi direct drive”. In a quasi direct drive actuator, a low gearing ratio is used, typically less than 10 to 1, which minimizes the amount of backlash and reflected inertia as observed at the output. Then, high rate electronic control of torque in the servo based on current and position feedback allows for dynamic manipulation of the spring and dampening of the resulting system.
Another option is a series elastic actuator, which uses a traditional high gear reduction servo with a mechanical spring or elastic mechanism inline with the load. Sometimes a separate motorized actuation mechanism can be used to vary the damping properties of the elastic element. This is in principle similar to the quasi direct drive approach, but suffers from a limited overall control bandwidth. Despite being “springy”, QDD servos are still able to have a very high effective mechanical control bandwidth, on the order of hundreds of hertz.
For the quad A1 to date, the compliance it exhibits is largely due to the qdd100’s internal control algorithms, and to a very minor extent, flexing in the mechanical structures of the quad A1 itself. This does work, and gives decent results.
The biggest limitation of solely using this approach, is that since the compliance is performed at the joint level, it has no knowledge of the current 3d configuration of the leg. The resulting compliance in 3D space is highly non-linear and depends upon where in configuration space the leg is at that point in time. For instance, if the back legs are configured to have the knee very bent, but the front legs are not, then the back knee needs a much larger restorative torque per unit rotation to have the same linear restorative force at the tip of the leg.
That results in artifacts like shown in the video at the bottom. When the robot falls with the legs not in an identical configuration, the robot ends up pitching or rolling depending upon how the compliance interacts with the current leg geometry.
In my original designs for the moteus controller, I had left a high rate “inter-leg” bus option in the design, where each controller could exchange IK information at the full control rate, so that all compliance could be performed in the 3D space, rather than in joint space. However, as the design progressed, and I failed to implement it, I dropped that capability to simplify and reduce costs.
Here, I ended up implementing something purely in software which doesn’t have the same level of performance as that system would have, but also doesn’t require additional dedicated high rate communication transceivers on every servo control board. The 3D PD controller is just run on the raspberry pi at the regular control update rate (400Hz currently). That makes the control flow look like this:
While this solution isn’t perfect, it does give better results in many scenarios. I applied some disturbances to the robot with either solely joint level controllers, or joint plus XYZ controllers. For the two cases, I tried to tune the controllers to a similar level of stiffness and damping to make the comparison as fair as possible. Walking is generally improved as well, even with just a constant compliance throughout the gait cycle.
https://shop.mjbots.com is now https://mjbots.com (don’t worry, the old site redirects)! The functionality is largely the same, you can still get your qdd100 actuators or moteus controllers. The biggest differences are 1) it looks slightly nicer, and 2) shipping rates are improved, and international shipping rates drastically so. For instance, DHL “Express” 2 day shipping to some points in Europe is now under $35 USD, whereas previously 2 day shipping was over $300. That is often cheaper than even USPS International Priority — which is typically 2-4 weeks.
I’ll be adding some more products over the next couple of weeks, and I wanted to make them as accessible to a worldwide audience as possible!
Now that I have the qdd100 servo in beta phase, the IMU working at full rate, and the quad A1 is moving around I’m getting closer to actually working to improve the gaits that the machine can execute. To date, the gaits I have used completely ignore the IMU and only use the feedback from the joints in order to maintain force in 3D. With tuning and on controlled surfaces this can work well, but if you go outside the happy regime, then it can undergo significant pitch and roll movements during the leg swing phase, which at best results in a janky walk, and at worst results in oscillation or outright instability.
There are also a number of as-yet-unidentified problems that seemingly cause the feet to not track the ground position properly, resulting in the feet slipping on the floor despite being nearly fully loaded.
To tackle all these new domains requires some improvements to my diagnostics infrastructure and tools. I’ll cover the improvements I’ve made in a few posts, since the work that has gone into it has covered a fair amount of ground. I’ll start with something I mostly completed back in the summer of 2019 and has the least direct impact, but gives at least a background for some of the other upcoming changes.
Super Mega Microbot since its inception in 2014 used a self-describing serialization and telemetry format that was loosely based on work I had done professionally previously at Bluefin Robotics and then Jaybridge Robotics. This format was then the basis for later work at Jaybridge and Toyota Research Institute. The basic idea breaks down like this:
The schema which describes the data and the data are separate entities
The schema is recorded alongside the data whenever it is written to persistent storage
The schema contains sufficient information to reconstruct a CSV or JSON like representation of the data with no additional meta-data
Structure tools can map a given on disk-schema to a possibly different in-memory one using a schema evolution algorithm
The data is serialized and stored in a manner which is very efficient to write at high rates from realtime processes
Compared to other serialization mechanisms, this has different trade-offs.
Formats like JSON, XML, either completely include the schema in each data instance, or include a large amount of self-describing information in each data instance that is not strictly necessary to represent it
Formats like protobuf, capnproto, flatbuffers, and SBE have a different tradeoff. They are geared towards performance, but largely also assume a single canonical source of schema data that is shared through an independent side channel and has a single linear revision history. This makes sense for server RPC, where client and server are each distributed (possibly different) versions of the schema and want to communicate without having to exchange it. They also include more metadata in the data stream than is strictly required many of them are more expensive to serialize or deserialize.
The closest to this work is Apache AVRO. It uses the same principle of separate schema and data, and expects the schema to be stored alongside the data. It also requires no code generation, which many of the above tools do require.
The unique pieces in this work over AVRO are that:
The data format is such that many common in-memory structures can simply be bit copied as serialized data with no further effort. Those that do require some manipulation still require no additional in-memory structures associated with serialization. This combines the properties of protobuf in that the serialization objects can be used as mutable state, with those of capnproto that allows zero cost serialization.
No recursion or pointers are supported, which renders the necessary code very simple. The entirety of the C++ serialization and deserialization library is only a few hundred lines of code and took less than a week overall to write, unit test, and debug over the 6 years I’ve been using it. It also functions perfectly fine in microcontroller-based embedded environments like the moteus controller.
The on-disk format is designed for rapid random seek access in time, assuming that small-ish records are written regularly.
The downsides are that it isn’t widely supported, isn’t optimized to handle single structures which have very large serialized representations, and the only language bindings aside from C++ are read only ones for python and TypeScript.
In future articles, I’ll describe a bit of the detail of the recently revised design, then go into the tools that use it.
It seems that I’m learning much about PCB design the very hard way. Back last year I wrote up my discovery of MLCC bias derating. Now I’ll share some of my experiences with MLCC cracking on the first production moteus controllers.
When I was first putting the production moteus controllers through their test and programming sequence, I observed a failure mode that I had yet to have observe in my career (which admittedly doesn’t include much board manufacturing). When applying voltage, I got a spark and puff of magic smoke from near one of the DC link capacitors on the left hand side. In the first batch of 40 I programmed, a full 20% failed in this way, some at 24V, and a few more at a 38V test. I initially thought the problem might have been an etching issue resulting in voltage breakdown between a via and an internal ground plane, but after examining the results under the microscope and conferring with MacroFab determined the most likely cause was cracking of the MLCCs during PCB depanelization.
Here’s a video describing the problem and potential solutions in way more detail than I’ll go into:
Needless to say, I hadn’t managed to see this failure in the 100 or so previous moteus controllers I’ve built, or I would have figured this out and resolved it!
For this first round of production controllers, I went and replaced every single capacitor near the edge of the board with a TDK variant that has internal soft termination, then tested them all at max voltage and a little beyond. Future revisions will use that variant of capacitor everywhere, as well as relocating the capacitors to reduce the mechanical stress they experience during manufacturing, handling, and installation.
Like with the fdcanusb, I built a programming and test fixture for the moteus controllers. The basic setup is similar to the fdcanusb. I have a raspberry pi with a touchscreen connected via USB to a number of peripherals. In this case, there is a STM32 programmer, a fdcanusb, and a label printer. Here though, unlike with the fdcanusb fixture, I wanted to be able to test the drive stage of the controllers and the encoders too.
My solution was to create a mechanical fixture that each board slots onto, with pogo pins that connect to test points for the phase outputs.
While it doesn’t make as good a connection as the solder through holes normally used to connect a motor, it is good enough to verify that the controller works. As a side-bonus, it also makes it trivial to test that the absolute magnetic encoder works properly.
This video shows how the programming and testing process works, and walks through testing a few boards.
This post will be short, because it is just re-implementing the functionality I had in my turrets version 1and 2, but this time using the raspberry pi as the master controller and two moteus controllers on each gimbal axis.
I have the raspberry pi running the primary control loop at 400Hz. At each time step it reads the IMU from the pi3 hat, and reads the current state of each servo (although it doesn’t actually use the servo state at the moment). It then runs a simple PID control loop on each axis, aiming to achieve a desired position and rate, which results in a torque command that is sent to each servo. Here’s the video proof!
To date, I’ve used the moteus controllers exclusively for joints in dynamic quadrupedal robots. However, they are a relatively general purpose controller when you need something that is compact with an integrated magnetic encoder. For the v3 of my Mech Warfare turret I’m using the moteus controllers in a slightly new configuration, with a gimbal motor, one for each of the pitch and yaw axes.
Gimbal motor theory and current sensing
From an electrical perspective, gimbal motors are not that all that different from regularly wound brushless outrunners. The primary difference being that they are wound with a much higher winding resistance. That enables them to be driven with a much lower current, at the expense of a lower maximum angular velocity. In this case, I’m using the GM3506 from iFlight which has a winding resistance of 6 ohms, that results in working currents being on the order of 2A maximum.
The moteus controllers are designed to drive motors with phase currents in the 20-60A range. To operate in current control mode, they use a current shunt resistor connected to a dedicated amplifier for each phase. The current sensing resistor determines the range of currents that can be accurately sensed. The resistor that the controllers have by default measures 0.5 milliohms, which gives a reasonable tradeoff between accuracy and power dissipation for 60A operation. However, for 2A operation, it results in pretty low resolution current feedback. Thus for this application, I substituted in 5 milliohm current shunts:
Control modes and noisy velocity
The other potential challenge, is that the velocity signal that can be derived from the AS5047 absolute magnetic encoder in the moteus controller is relatively noisy when operated with no gearbox reduction. That limits how much derivative gain can be used in a position control loop. You could get around that by incorporating more plant knowledge in a filter or state observer, but that shouldn’t be necessary here.
Fortunately for this application, I won’t be controlling position based on the absolute magnetic encoder, but instead based on the inertial measurement unit in the primary controller. The absolute magnetic encoder will be used solely for performing the DQ transform to implement torque control, for which the noise in velocity is irrelevant.
Next up is making these controllers actuate the turret.
Another of the tasks I’ve set for myself with regards to future Mech Warfare competitions is redesigning the turret. The previous turret I built had some novel technical features, such as active inertial gimbal stabilization and automatic optical target tracking, however it had some problems too. The biggest one for my purposes now, was that it still used the old RS485 based protocol and not the new CAN-FD based one. Second, the turret had some dynamic stability and rigidity issues. The magazine consisted of an aluminum tube sticking out of the top which made the entire thing very top heavy. The 3d printed fork is the same I one I had made at Shapeways 5 years ago. It is amazingly flexible in the lateral direction, which results in a lot of undesired oscillation if the base platform isn’t perfectly stable. I’ve learned a lot about 3d printing and mechanical design in the meantime (but of course still have a seemingly infinite amount more to learn!) and think I can do better. Finally, cable management between the top and bottom was always challenging. You want to have a large range of motion, but keeping power and data flowing between the two rotating sections was never easy.
My concept with this redesign is twofold, first make the turret be basically an entirely separate robot with no wires connecting it to the main robot and second, try to use as many of the components from the quad A1 as I could to demonstrate their, well, flexibility. Thus, this turret will have a separate battery, power distribution board, raspberry pi, pi3 hat, and a moteus controller for each axis of motion. These are certainly overkill, but hey, the quad A1 can carry a lot of weight.
The unique bits will be a standalone FPV camera, another camera attached to the raspberry PI for target tracking, a targeting laser, and the AEG mechanism, including a new board to manage the firing and loading functions.