Tag Archives: moteus

Dealing with stator magnetic saturation

robots, , , ,

In my previous experiments demonstrating torque feedback (full rate inverse dynamics, ground truth torque testing), I’ve glossed over the fact that as the stator approaches magnetic saturation, the linear relationship between torque and current breaks down. Now finally I’ll take at least one step towards allowing moteus to accurately work in the torque domain as motors reach saturation.

Background

The stator in a rotor consists of windings wrapped around usually an iron core. The iron in the core consists of lots of little sub-domains of magnetized material, that normally are randomly oriented resulting in a net zero magnetic field. As current is applied to the windings, those domains line up, greatly magnifying the resulting magnetic field. Eventually most of the sub-domains are aligned, at which point you don’t get any more magnifying effect from the iron core. In this region, the stator is said to be “saturated”. You can read about it in much more depth on wikipedia or with even more detail here. The end result is a curve of magnetic field versus applied current that looks something like this:

To date, moteus assumes that you are operating completely in the “Linear” region, where the torque and current are linearly related.

Operating in the Rotation Region

To operate in the “rotation” region I ended up using the following formula:

\tau = K_T * I_c + ts * log2(1 + (I - I_c) * is)

Where I is the input current, K_T is the motor torque constant, ts, I_c and is are three constants that I fit to measured torque data. With some approximations, this can be calculated relatively efficiently on the STM32G4 that drives the moteus controller, adding only a microsecond to the overall loop time to go in both directions.

I then ran a torque sweep with my load-cell fixture from before, and sure enough, the input and output torque match much better now across the entire range of operation, despite the fact that the phase current needs to start growing very rapidly near the top end:

Testing qdd100 stator windings

robots, , , ,

My initial design torque for the qdd100 was a little over 17 Nm. However, when I did my first ground truth torque testing, I found that some servos had a lower maximum torque than I had specified. While working to diagnose those, I built a qdd100 that used an alternate stator winding of 105Kv instead of the 135Kv that are in all the beta units. The Kv rating of a stator describes how fast the motor will spin for a given applied voltage. If you assume the same amount of copper mass of wiring, a lower Kv will mean that there are thinner wires that wrap around the stator more turns (or fewer wires in parallel). A higher Kv will have thicker wires with fewer overall turns.

On paper, if you assume a perfect controller, this shouldn’t make much of a difference. The same input power should be required for the same output torque. The only differences should come into play once you have a controller with either a limited maximum voltage or a limited maximum current. The higher Kv motor will be able to go faster given a fixed maximum voltage, and the lower Kv motor will have more torque for a given maximum current.

I wanted to verify that this was true as part of my evaluation to identify the cause of my decreased torque, so I used a slightly upgraded torque testing fixture:

For now, I rigged up the world’s cheapest load cell from amazon to a Nucleo configured to report the load in grams over the serial port. I also wired up my Chroma power supply over USB using the linux USBTMC driver. With those two things hooked up, I was able to run tests that sweeped across torque commands, while recording output torque, phase current, and input power.

At higher torques, the input power was pretty sensitive to the temperature of the windings — hotter windings increased the resistance, which increased the power required to achieve a given phase current, thus my plot isn’t perfect as it was grabbed over several different runs. For the highest power samples I couldn’t use my Chroma, as it is limited to around 600W. Thus those samples don’t record the input power.

Plotting the input power vs output torque on the same chart shows that indeed, modulo some measurement error, they are the same for the two stators:

So, this experiment reaffirmed my understanding of stator magnetics and confirmed that the stator winding was not the cause of my decreased torque.

micro-BOM management

robots, , , ,

I’ve now built 3 or 4 complete quad A1 style robots depending upon how you look at it. Each was somewhat of a one-off, incrementally modified over time as I discovered failure modes and improved the design. Before starting to serially build quad A1 style robots, I wanted to get a better understanding of how much actually goes into making one. The quad A1 has a fair number of sub-assemblies, custom PCBs, harnesses, and assembly steps that go into its production. During previous builds, I kept running into problems where I would run out of some component, fastener, or raw material unexpectedly, then have to wait for its lead time before I could continue.

I’m currently using a simplified Kanban inventory tracking system where I re-order parts and components when their inventory level reaches a critical threshold. However, given the discontinuous nature of my production, setting those levels is really hard. I didn’t know, for instance, just how many M3x8 bolts were necessary between the chassis, legs, and all the other sub-assemblies, since I had never really built multiple of all of them at one time before.

Incremental improvement

To make my life a little easier, I’ve started on a “micro-BOM” management solution. It currently is just a simple C++ application that reads a tree of JSON5 (mostly so I can use the obvious comment style) files on disk. Each JSON5 file describes a single stock item, what sub-components it consists of, and what it takes to acquire it in terms of cash, assembly time, 3d printer time, ordering information, and assembly procedures.

{
  "name" : "quada1_leg_lower",
  "version" : "20200701.0",
  "uuid" : "ad3f6ce2-aa13-42a2-8a4c-2d4b4e799ab0",
  "components" : [
    { "name" : "quada1_leg_foot", "qty" : 1 },  // the squash ball
  ],
  "resources" : [
    { "mk3s" : 7.5, "petg_black" : 71.54, "hours_tech1" : 0.1 },
  ],
  "procure" : {
    "print" : "20200625-lower_leg_and_bracket.3mf",
  },
}

That tool can then follow the hierarchy to count up how many stock units of each type are necessary for any top level item, and how many “resources” are necessary to make that happen.

Thus after entering in ~85 separate “stock units”, I now know that the current version of a full quad A1 requires approximately 300 hours of MK3s printer time between all of its assemblies and sub-assemblies, 132 heat set inserts, and 310 fasteners of various types (assuming the qdd100s are pre-assembled)!

Next steps

Given that I have relatively deep sub-assembly trees, and that I expect to be keeping some stock of all of them, I want to make this tool inventory-aware. That way it knows I already have N leg sub-assemblies in stock or Y qdd100s already built, and thus can tell me how many more bolts and such are necessary to build a full robot.

I’d also like to be able to export the results from this to replace the human-readable bom.txt entries sprinkled throughout git. That will make them both more accurate, and easier to maintain.

High speeds with the moteus controller

robots,

Someone contacted me not too long ago who wanted to use the moteus controller, but wasn’t sure if it would be able to hit their target mechanical velocity of 6000rpm. I honestly wasn’t either, so I tested it. After a quick firmware fix, the devkit motor when run at 34V seems to be able to do it no problem.

It should be noted that the current firmware assumes you are within a thousand or so revolutions of 0. You can exceed that pretty quickly running at 120 revolutions per second!

mjbots Monday: New lower prices

robots, , ,

One of my goals with mjbots is to make building dynamic robots more accessible to researchers and enthusiasts everywhere. To make that more of a reality, I’m lowering the prices in a big way on the foundational components of brushless robotic systems, the moteus controller and qdd100 servo.

OldNew
moteus r4.3 controller$119$79
moteus r4.3 devkit$199$159
qdd100 beta$549$429
qdd100 beta devkit$599$469

Don’t worry, if you purchased any of these in the last month, you should be getting a coupon in your email equivalent to the difference.

Happy building!

Cartesian leg PD controller

robots, , , , , , ,

As I am working to improve the gaits of the mjbots quad A1, one aspect I’ve wanted to tackle for a long time is improving the compliance characteristics of the whole robot. Here’s a small step in that direction.

Existing compliance strategy

The quad A1 uses qdd100 servos for each of its joints. The “qdd” in qdd100 stands for “quasi direct drive”. In a quasi direct drive actuator, a low gearing ratio is used, typically less than 10 to 1, which minimizes the amount of backlash and reflected inertia as observed at the output. Then, high rate electronic control of torque in the servo based on current and position feedback allows for dynamic manipulation of the spring and dampening of the resulting system.

Another option is a series elastic actuator, which uses a traditional high gear reduction servo with a mechanical spring or elastic mechanism inline with the load. Sometimes a separate motorized actuation mechanism can be used to vary the damping properties of the elastic element. This is in principle similar to the quasi direct drive approach, but suffers from a limited overall control bandwidth. Despite being “springy”, QDD servos are still able to have a very high effective mechanical control bandwidth, on the order of hundreds of hertz.

For the quad A1 to date, the compliance it exhibits is largely due to the qdd100’s internal control algorithms, and to a very minor extent, flexing in the mechanical structures of the quad A1 itself. This does work, and gives decent results.

Limitations

The biggest limitation of solely using this approach, is that since the compliance is performed at the joint level, it has no knowledge of the current 3d configuration of the leg. The resulting compliance in 3D space is highly non-linear and depends upon where in configuration space the leg is at that point in time. For instance, if the back legs are configured to have the knee very bent, but the front legs are not, then the back knee needs a much larger restorative torque per unit rotation to have the same linear restorative force at the tip of the leg.

That results in artifacts like shown in the video at the bottom. When the robot falls with the legs not in an identical configuration, the robot ends up pitching or rolling depending upon how the compliance interacts with the current leg geometry.

A “fix”

In my original designs for the moteus controller, I had left a high rate “inter-leg” bus option in the design, where each controller could exchange IK information at the full control rate, so that all compliance could be performed in the 3D space, rather than in joint space. However, as the design progressed, and I failed to implement it, I dropped that capability to simplify and reduce costs.

Here, I ended up implementing something purely in software which doesn’t have the same level of performance as that system would have, but also doesn’t require additional dedicated high rate communication transceivers on every servo control board. The 3D PD controller is just run on the raspberry pi at the regular control update rate (400Hz currently). That makes the control flow look like this:

Results

While this solution isn’t perfect, it does give better results in many scenarios. I applied some disturbances to the robot with either solely joint level controllers, or joint plus XYZ controllers. For the two cases, I tried to tune the controllers to a similar level of stiffness and damping to make the comparison as fair as possible. Walking is generally improved as well, even with just a constant compliance throughout the gait cycle.

New mjbots.com

robots, , ,

https://shop.mjbots.com is now https://mjbots.com (don’t worry, the old site redirects)! The functionality is largely the same, you can still get your qdd100 actuators or moteus controllers. The biggest differences are 1) it looks slightly nicer, and 2) shipping rates are improved, and international shipping rates drastically so. For instance, DHL “Express” 2 day shipping to some points in Europe is now under $35 USD, whereas previously 2 day shipping was over $300. That is often cheaper than even USPS International Priority — which is typically 2-4 weeks.

I’ll be adding some more products over the next couple of weeks, and I wanted to make them as accessible to a worldwide audience as possible!

Updated serialization library (diagnostics part 1)

robots, , , , , ,

Now that I have the qdd100 servo in beta phase, the IMU working at full rate, and the quad A1 is moving around I’m getting closer to actually working to improve the gaits that the machine can execute.  To date, the gaits I have used completely ignore the IMU and only use the feedback from the joints in order to maintain force in 3D.  With tuning and on controlled surfaces this can work well, but if you go outside the happy regime, then it can undergo significant pitch and roll movements during the leg swing phase, which at best results in a janky walk, and at worst results in oscillation or outright instability.

There are also a number of as-yet-unidentified problems that seemingly cause the feet to not track the ground position properly, resulting in the feet slipping on the floor despite being nearly fully loaded.

To tackle all these new domains requires some improvements to my diagnostics infrastructure and tools.  I’ll cover the improvements I’ve made in a few posts, since the work that has gone into it has covered a fair amount of ground.  I’ll start with something I mostly completed back in the summer of 2019 and has the least direct impact, but gives at least a background for some of the other upcoming changes.

Telemetry format

Super Mega Microbot since its inception in 2014 used a self-describing serialization and telemetry format that was loosely based on work I had done professionally previously at Bluefin Robotics and then Jaybridge Robotics.  This format was then the basis for later work at Jaybridge and Toyota Research Institute.  The basic idea breaks down like this:

  • The schema which describes the data and the data are separate entities
  • The schema is recorded alongside the data whenever it is written to persistent storage
  • The schema contains sufficient information to reconstruct a CSV or JSON like representation of the data with no additional meta-data
  • Structure tools can map a given on disk-schema to a possibly different in-memory one using a schema evolution algorithm
  • The data is serialized and stored in a manner which is very efficient to write at high rates from realtime processes

Compared to other serialization mechanisms, this has different trade-offs.

  • Formats like JSON, XML, either completely include the schema in each data instance, or include a large amount of self-describing information in each data instance that is not strictly necessary to represent it
  • Formats like protobuf, capnproto, flatbuffers, and SBE have a different tradeoff.  They are geared towards performance, but largely also assume a single canonical source of schema data that is shared through an independent side channel and has a single linear revision history.  This makes sense for server RPC, where client and server are each distributed (possibly different) versions of the schema and want to communicate without having to exchange it.  They also include more metadata in the data stream than is strictly required many of them are more expensive to serialize or deserialize.
  • The closest to this work is Apache AVRO.  It uses the same principle of separate schema and data, and expects the schema to be stored alongside the data.  It also requires no code generation, which many of the above tools do require.

The unique pieces in this work over AVRO are that:

  • The data format is such that many common in-memory structures can simply be bit copied as serialized data with no further effort.  Those that do require some manipulation still require no additional in-memory structures associated with serialization.  This combines the properties of protobuf in that the serialization objects can be used as mutable state, with those of capnproto that allows zero cost serialization.
  • No recursion or pointers are supported, which renders the necessary code very simple.  The entirety of the C++ serialization and deserialization library is only a few hundred lines of code and took less than a week overall to write, unit test, and debug over the 6 years I’ve been using it.  It also functions perfectly fine in microcontroller-based embedded environments like the moteus controller.
  • The on-disk format is designed for rapid random seek access in time, assuming that small-ish records are written regularly.

The downsides are that it isn’t widely supported, isn’t optimized to handle single structures which have very large serialized representations, and the only language bindings aside from C++ are read only ones for python and TypeScript.

In future articles, I’ll describe a bit of the detail of the recently revised design, then go into the tools that use it.

 

More MLCC learning

robots, , , , ,

It seems that I’m learning much about PCB design the very hard way.  Back last year I wrote up my discovery of MLCC bias derating.  Now I’ll share some of my experiences with MLCC cracking on the first production moteus controllers.

When I was first putting the production moteus controllers through their test and programming sequence, I observed a failure mode that I had yet to have observe in my career (which admittedly doesn’t include much board manufacturing).  When applying voltage, I got a spark and puff of magic smoke from near one of the DC link capacitors on the left hand side.  In the first batch of 40 I programmed, a full 20% failed in this way, some at 24V, and a few more at a 38V test.  I initially thought the problem might have been an etching issue resulting in voltage breakdown between a via and an internal ground plane, but after examining the results under the microscope and conferring with MacroFab determined the most likely cause was cracking of the MLCCs during PCB depanelization.

dsc_0415
A very obviously cracked capacitor

Here’s a video describing the problem and potential solutions in way more detail than I’ll go into:

Needless to say, I hadn’t managed to see this failure in the 100 or so previous moteus controllers I’ve built, or I would have figured this out and resolved it!

For this first round of production controllers, I went and replaced every single capacitor near the edge of the board with a TDK variant that has internal soft termination, then tested them all at max voltage and a little beyond.  Future revisions will use that variant of capacitor everywhere, as well as relocating the capacitors to reduce the mechanical stress they experience during manufacturing, handling, and installation.

Programming and testing moteus controllers

robots, , , ,

Like with the fdcanusb, I built a programming and test fixture for the moteus controllers.  The basic setup is similar to the fdcanusb.  I have a raspberry pi with a touchscreen connected via USB to a number of peripherals.  In this case, there is a STM32 programmer, a fdcanusb, and a label printer.  Here though, unlike with the fdcanusb fixture, I wanted to be able to test the drive stage of the controllers and the encoders too.

My solution was to create a mechanical fixture that each board slots onto, with pogo pins that connect to test points for the phase outputs.

dsc_0428

While it doesn’t make as good a connection as the solder through holes normally used to connect a motor, it is good enough to verify that the controller works.  As a side-bonus, it also makes it trivial to test that the absolute magnetic encoder works properly.

This video shows how the programming and testing process works, and walks through testing a few boards.