Tag Archives: rpi

Spread spectrum integration

I’ve been developing a new bi-directional spread spectrum radio to command and control the mjbots quad robot.  Here I’ll describe my first integration of the protocol into the robot.

To complete that integration, I took the library I had designed for the nrfusb, and ported it to run on the auxiliary controller of the pi3 hat.  This controller also controls the IMU and an auxiliary CAN-FD bus.  It is connected to one of the SPI buses on the raspberry pi.  Here, it was just a matter of exposing an appropriate SPI protocol that would allow the raspberry pi to receive and transmit packets.

Slightly unfortunately, this version of the pi3hat does not have interrupt lines for any of the stm32s.  Thus, I created a multiplexed status register that the rpi can use to check which of the CAN, IMU, or RF has data pending.  Then I slapped together a few registers which allowed configuring the ID and reading and writing slots and their priorities.

Then I refactored things around on the raspberry pi side so that one core would keep busy polling for one of those things to become available.  So far, for the things which access SPI, I’ve been putting them locked to an isolcpu cpu to get improved SPI timing.  Eventually, once I have interrupt lines, I might consolidate all of these down to a single core.  That, plus defining an initial mapping between the controls and slots resulted in:

Finally, I created a very simple GL gui application which connects to an nrfusb and a joystick.  It uses Dear ImGui to render a few widgets and glfw to window and read the joystick.

2020-04-03-145917_1280x720_scrot

While I was at it, I finally updated my joystick UI to make gait selection a bit faster, and got the robot to do a better job of switching out of the walk gait.  Thus the following video showing all of that hooked together.

Spread spectrum implementation

With a protocol design in hand, the next step was to go and implement it.  My goal was to produce a library which would work on the nrfusb, and also on the auxiliary stm32g4 on the mjbots pi3 hat.  In this first implementation pass however, I only worked with the nrfusb as both transmitter and receiver.

While developing this, I had more than my share of “huh” moments working from the datasheet and with the components.  To begin with, the initial nrf24l01+ modules I got were all Chinese clone ones.  While I was having problems getting auto acknowledgement to work, I discovered that the clones at a minimum were not compatible with genuine Nordic devices.  Thus I reworked genuine parts into the modules I had:

dsc_0376

dsc_0375
A Nordic clone about to be removed

That didn’t solve any of my immediate problems, but the subsequent modules I got all had genuine chips so it was useful that they all were compatible.

The other more annoying problems are somewhat obvious in hindsight.  For a transmitter to be able to successfully receive an automatic acknowledgment from a receiver, not only does the ID need to be configured in the appropriate RX_ADDR register, but EN_RXADDR also needs to have the correct bit set.  I had assumed that was only required for slave devices as there was no mention of it in any of the Enhanced Shockburst flow charts or setup procedures for transmitters or auto acknowledgment.

The second annoyance, was that when in receiver mode, switching channels seems to kinda work a little bit for some channels even with CE held high, but to be reliable you have to pull CE low and put the unit in standby mode while changing channels.

With those problems (and some others) resolved, I have a reliable bidirectional link that is ultimately tweakable.  Next I’ll integrate this into the quad A1 to actually control the robot and monitor its telemetry.

 

nrfusb

In order to bring up the final piece of the raspberry pi 3 hat, the nrf24l01+, I wanted a desktop development platform that would allow for system bringup and also be useful as a PC side transmitter.  Thus, the nrfusb:

dsc_0367

Similar to the fdcanusb, it is just an STM32G474 on the USB bus, although this has a pin header for a common nrf24l01+ form factor daughterboard.

The next steps here are to get this working at all, then implement a spread spectrum bidirectional protocol for control and telemetry.

Building the quad A1

Now that I have a bunch of the mk2 servos set and ready to go, a new leg design, a new power distribution board to power them, and a raspberry pi3 hat to communicate with them, I built a new quadruped!  I’m calling this the mjbots quad A1, since basically everything is upgraded.

dsc_0362

After I initially assembled the new legs onto the chassis, I realized I had the geometry slightly off and there was some interference through part of the shoulder rotation.  I made up new printed parts and replaced everything in front of the camera.  Thus, watch some high speed robot surgery:

The quad A1’s first job is to validate the new moteus controller in the quadrupedal configuration, after which I’ll use it as the primary development platform to get all my gait work done.

 

Attitude estimation for pi3 hat

Now that the IMU is functioning, my next step is to use that to produce an attitude estimate.  Here, I dusted off my unscented Kalman filter based estimator from long ago, and adapted it slightly to run on an STM32.  As before, I used a UKF instead of the more traditional EKF not because of its superior filtering performance, but because of the flexibility it allows with the process and measurement functions.  Unlike the EKF, the UKF is purely numerical, so no derivation of Jacobians is necessary.  It turns out that even an STM32 has plenty of processing power to do this for things like a 7 state attitude filter.

One problem I encountered, was by default I have been building everything for the STM32 with the “-Os” optimization level.  Unfortunately, with Eigen linear algebra routines, that is roughly 4x slower than “-O3”.  Doubly unfortunate, just using copts at the rule level or --copts on the command line didn’t work.  bazel doesn’t let you control the order of command line arguments very well, and the -Os always ended up *after* any of the additional arguments I tried to use to override.  To get it to work, I had to navigate some bazel toolchain mysteries in rules_mbed in order to allow build rules to specify if they optionally want the higher optimization instead of optimizing for size.  I’m pretty sure this is not exactly what the with_features mechanism in toolchain’s feature rule is for, but it let me create a feature called speedopt which turns on -O3 and turns off -Os.  The final result is at rules_mbed/530fae6d8

To date, I’ve only done some very zeroth order performance optimization.  I spent 15 minutes parameter tuning, making sure that the covariances updated to approximately the correct levels and  I added a simple filter to reject accelerometer updates during dynamic motion.  I did just enough runtime performance to get an update down to around 300us, which is just fine for a filter intended to run at 1kHz.  More will remain as future work.

Here’s a plot from a quick sanity check, where I manually rolled the device in alternating directions, then pitched it in alternating directions.  (When pitching, it was on a somewhat springy surface, thus the ringing).

20200305-attitude-validation

The pitch and roll are plenty smooth, although they look to perhaps not returning exactly to their original position.  At some point, I will do a more detailed qualification to dial in the performance.

 

Bringing up CAN on the quad pi3 hat

After getting the power to work, the next step in bringing up the new quad’s raspberry pi interface board is getting the FDCAN ports to work.  As described in my last roadmap, this board has multiple independent FDCAN buses.  There are 2 STM32G4’s each with 2 FDCAN buses so that every leg gets a separate bus.  There is a 5th auxiliary bus for any other peripherals driven from a third STM32G4.  All 3 of the STM32G4’s communicate with the raspberry pi as SPI slaves.

Making this work was straightforward, if tedious.  I designed a simple SPI based protocol that would allow transmission and receipt of FD-CAN frames at a high rate in a relatively efficient manner, then implemented that on the STM32s.  On the raspberry pi side I initially used the linux kernel driver, but found that it didn’t give sufficient control over hold times during the transmission.  Since the SPI slave is implemented in software, I needed to leave sufficient time after asserting the chip select and after transmitting the address bytes.  The kernel driver gives no control over this at all, so I resorted to directly manipulating the BCM2837s peripheral registers and busy loop waiting in a real time thread.

dsc_0347
A lot of scope probes!

After a decent supply of bugs were squashed, I got to a point where the host could send off 12 queries to all the servos with the four buses all being used simultaneously, then collating the responses back.  I haven’t spent much time optimizing the cycle time, but the initial go around is at around 1.0ms for a query of all 12 devices which is about 1/3 of the 3.5ms I had in the previous single-bus RS485 version.

20200226-leg-transaction

Here’s a scope trace of a full query cycle with 2 of the 4 CAN buses on the top, and the two chip selects on the bottom.  Woohoo!

Bringing up the IMU on the pi3 hat

The next peripheral to get working on the quad’s raspberry pi interface board is the IMU. When operating, the IMU will primarily be used to determine attitude and angular pitch and roll rates.  Secondarily, it will determine yaw rate, although there is no provision within the IMU to determine absolute yaw.

To accomplish this, the board has a BMI088 6 axis accelerometer and gyroscope attached via SPI to the auxiliary STM32G4 along with discrete connections for interrupts.  This chip has 16 bit resolution for both sensors, decent claimed noise characteristics, and supposedly the ability to better reject high frequency vibrations as seen in robotic applications.  I am currently running the gyroscope at 1kHz, and the accelerometer at 800Hz.  The IMU is driven off the gyroscope, with the accelerometer sampled whenever the gyroscope has new data available.

My first step was just to read out the 6 axis values at full rate to measure the static performance characteristics.  After doing that overnight, I got the following Allan Variance plot.

20200304-bmi088-allan-variance

That gives the angular random walk at around 0.016 dps / sqrt(Hz) with a bias stability of around 6.5 deg/hr.  The angular random walk is about what is specified in the datasheet, and the bias is not specified at all, but this seems really good for a MEMS sensor.  In fact, it is good enough I could probably just barely gyrocompass, measuring the earth’s rotation, with a little patience.  The accelerometer values are shown there too, and seem fine, but aren’t all that critical.

Next up is turning this data into an attitude and rate estimate.

Bringing up power on the quad pi3 hat

The first thing I needed to get working on the new quad’s raspberry pi3 hat, was the input DC/DC power converter.  One of the main functions of this board is to take the main DC bus voltage of around 20V, and provide the raspberry pi with 5V power.

In the previous iteration of this board, it was limited to an recommended maximum voltage of around 24V.  As with all the components in my hardware revisions I aimed to support a higher input voltage.  Here I switched parts to the Diodes AP64351 so that I could get to a recommended maximum voltage of 32V (the part’s absolute max is 40V).

Normally, bringing up power isn’t all that interesting.  Either it works, or some pin is obviously connected incorrectly and it doesn’t.  However here, I had different behavior.  When first powering on the device, it kinda flickered the output to 5V or less maybe once every second or so.  While probing with the multimeter, I found that when I probed the soft start selection pin all of a sudden it started to seemingly work!  Assuming the probe’s input impedance must be enough to do something, I soldered on an SMD resistor in parallel with the soft start selection capacitor (after trying more capacitance) and got it to work a little bit, but it was still flaky, cutting out whenever the raspberry pi started to draw significant current during the boot process.

dsc_0335

The second instance of the board exhibited similar symptoms, except there I managed to accidentally short the soft start selection pin to 18V, likely toasting it.  However, surprisingly, that seemed to get the chip into a working state!

After much thought (and I should have noticed it in the picture above), I discovered I had managed to populate the incorrect part, the AP64350, not the AP64351.  The x50 version is mostly pin compatible, but has a frequency selection pin where the x51 has a soft start selection pin.

Replacing that chip with the desired part got everything working as expected!

 

New quad raspberry pi interface board

With the new FD-CAN based moteus controllers I need a way for the raspberry pi to communicate with them.  Thus I’ve got a new adapter board in house that I’m bringing up:

dsc_0339

This one has 5 independent FD-CAN channels, an IMU, a port for an nrf2401l RF transceiver as well as a buck converter to power the computer from the main battery bus.

The prototypes were largely constructed by MacroFab, although I did the Amass connectors and the STM32s because supply chain issues prevented me from getting those parts to MacroFab in time.

Next I’ll start bringing up the various pieces!

mjbots quad A0: October 2019 Roadmap

My last video gave an overview of what I’ve accomplished over the past year.  Now, let me talk about what I’m planning to work on going forward:

I intend to divide my efforts into two parallel tracks.  The first is to demonstrate increased capabilities and continue learning with the existing quad A0, and second is to design and manufacture the next revision of all its major components.

New capabilities and learning

The first, and most important capability I want to develop is an improved gait and locomotion system.  While the moteus servos in the quad A0 are capable of high rate compliant control, the gait engine that I’m using now is still basically the same one that I made for the HerkuleX servos 5 years ago.  It just commands open loop positions to each of the servos and uses no feedback from the platform at all.  This severely limits what the robot can do.  For instance, if the terrain is not level, legs will drag on the ground or it will not walk at all.  The maximum speed is relatively slow and achieving it requires careful tuning of servo-level gains.  While it is more robust than nearly any other open loop 4 legged walker while standing up, even small disturbances can cause it to fall over.

At a minimum, I’d like to switch to controlling the position and force of the endpoint of each leg and switch to a gait engine that takes into account the actual position of each joint during the gait cycle.  That should enable it to maintain good ground contact with all feet that are intended to be in a stance position.  Now, often feet will skip around, as the servo gains are all relatively high to account for the lack of feedback.

The next level would be to take into account an IMU to ensure balanced feet lift offs.  While I have an IMU in this configuration, it currently isn’t usable due to my improvised attempts to improve the overall system update rate.  I’ll need to update some of the hardware to get an IMU again before I work on this part.

New hardware revisions

I have learned a lot with the moteus controller, servo, and the quad A0 over the last year.  I’d like to take that learning and update the components to achieve:

  • Manufacturability: Many of the sub components now are very labor intensive to produce or assemble.
  • Cost: I have in many cases made things much more expensive than they needed to be in order to shave a bit of time off the prototyping process.  Also, I have enough confidence now to get larger volumes of parts, which further reduces cost.
  • Capability: There are many easy areas where the system performance can be greatly improved.

My overall goals are to make a system that is capable and affordable — to provide an ecosystem of parts and systems for researchers and hobbyists to use.  That means keeping everything open source, and providing access to all subcomponents and designs while still being able to provide a complete system that is capable out of the box.

Here’s a quick rundown of the various areas I’m tackling:

Moteus controller

My next revision of this controller will be a slightly bigger change, in that the form factor, the microcontroller, and the connectors used will all be altered.  I’m planning on switching to an STM32G4 for the controller, using daisy-chained connectors for power in addition to communications, switching to FD-CAN for communications, and reworking the form factor to enable all the primary connectors to exit from one edge of the servo.

Also, I’ll take this opportunity to design for automated end of line test.  That means I will leave enough probe points such that a test fixture can validate the correct operation of a board without intervention.

I don’t anticipate the performance to change a whole lot, although I may increase the allowed input voltage.  Even so this will still be a very capable board, with 3 phase current sensing, >=50A peak phase current, >= 400W peak power and a switching and internal control loop rate of >=40kHz.

Moteus servo

The first round of gearbox servos went through many revisions to achieve something that was functional.  That said, while functional, they are fragile, and require about a full man day of assembly time per servo.  I’m working to get pre-production volumes of around 100 units for all the components such that no post machining will be required and the servos can just be assembled together.

While I’m at it, I intend on switching everything to metal from 3d printed plastic, and reducing the overall dimensions.  It is likely the mass will increase a little from the current 410g, but I’m hoping that it won’t be that much compared to the increased strength and rigidity.

Junction board / raspberry pi 3 hat / pre-charge board

I intend to rework the architecture behind the power and data distribution internal to the quad A0.  Currently there is a raspberry pi3 hat, which merely powers the raspberry pi and provides an RS485 interface.  Then a “junction board” splits out the RS485 bus into four separate connectors as well as splitting out power to all four legs and housing the IMU.  Finally, a separate “pre-charge” board ensured that the servos can have power applied safely.

I intend on moving all data distribution and the IMU onto the board that mounts on the host computer.  Thus it will have power in, an IMU, and 4 (or maybe more) CAN bus outputs.  Then, a separate power distribution board will include the pre-charge circuit as well as connectors for power for all four legs.  Simultaneously, I plan on adding an actual power switch, so that I don’t have to keep pulling the battery in and out to cut power.

Chassis

The quad A0 chassis is a single monolithic 3D printed block.  Actually attaching legs to this is nigh impossible.  It takes a long time, and not all of the joining screws can actually be used because of interference with other pieces.  Also, using the Ryobi style batteries results in a lot of dead space due to the mounting stud.  I am planning on switching this chassis to be printed in multiple components that are assembled together with screws and threaded inserts and switch to a battery form factor that is more compact.

That should also allow for mounting the primary computer inside the chassis, and still leave enough room for a bigger computer, like an NVIDIA Jetson nano.

Legs

I’ve already had one failure in the current iteration of the leg that indicates some redesign is necessary.  Also, the two primary brackets that connect the shoulder to the upper leg, and upper leg to lower leg should be made from aluminum instead of being 3d printed.  Those brackets had to be really big and printed in odd orientations in order to be strong enough, and they could be a lot lighter, more convenient, and more cost effective in metal.

Looking forward

Thanks for all of your positive feedback so far, and I look forward to more exciting times executing these plans!