When commanding a moteus controller, many of the registers defined in the reference documentation give explicit semantics when the value is passed as either a floating point NaN value or the “maximally negative integer” for integer encodings. Those that do not specify a meaning for NaN are “undefined” and anything can happen. For the most part, this isn’t a problem, but in one specific case users were repeatedly caught off guard.
Three common registers often set in position mode, (also accessible by the unlabeled first three arguments to “d pos”), are 0x020 “target position”, 0x021 “target velocity”, and 0x025 “maximum torque”. Target position has a defined meaning when set to NaN: the controller assumes that the target position is the current target position, or the sampled position if the controller is not yet in position mode. Maximum torque also has a defined meaning when set to NaN: it then uses the maximum torque as defined by the maximum phase current limit. However, target velocity had no such definition, and in practice when used, resulted in the controller becoming mostly non-functional.
To remedy that, as of release 2023-09-26, moteus will treat a velocity command of NaN the same as a velocity command of 0.0. That is what nearly all users of it intended from my discussions, and I wasn’t able to come up with any more useful semantics after a decent amount of time thinking about it.
No changes are required in user code, this just means that something which would previously result in a broken system, will now work and likely do what the user expected.
As of release 2023-09-26, moteus controllers can now use CUI AMT21 series encoders for any encoder source. CUI’s AMT21 series encoders are rugged, with resolution up to 14 bits, and since they use RS485 for communication can be located a significant distance from the motor driver if necessary. Setting one up is easy with the moteus-n1, which is what I’ll cover here.
Hardware
The moteus N1 has JST GH-6 connector labeled “RS422” with all the pins necessary to power and communicate with an AMT21. Since the AMT21 is RS485, not RS422, it is required to tie the A and Y pins together and the B and Z pins together. An easy option is to do so in the harness, either by crimping multiple wires into the Molex terminal for the AMT21, or by splicing wires. The desired schematic looks like:
Software
With the moteus-n1, the software configuration is straightforward, and basically the same as for the AksIM-2.
Once the aux1 UART is configured, then it can be selected as the source for any of the motor_position sources and selected for either commutation sensing, output/load sensing, or as an auxiliary sensor. For instance, to use it as the primary encoder instead of the onboard one, you would do:
motor_position.sources.0.aux_number 1
motor_position.sources.0.type 2 # uart
motor_position.sources.0.cpr 16384 # AMT21 devices report as 14 bit
motor_position.commutation_source 0
motor_position.output.source 0
moteus-r4
As with the AksIM-2, a moteus-r4 can be used to drive the CUI AMT21 series, but additional interface hardware is required. An example schematic can be found in the “RLS AksIM-2” section of the initial “worked examples” post.
As of release 2023-09-26, moteus r4 and moteus-n1 now both support the MA732 as an external SPI encoder. The magnetic sensing performance of the MA732 is normally a bit worse than the AS5047P that is used as the onboard encoder with moteus, but it has two possible advantages.
The first, is that it is much smaller, using a 3mm x 3mm QFN package. The second is that it supports off-axis applications. You will need to read the datasheet, and set the 'aux?.spi.bct' configuration parameter correctly, but this can enable operation in interesting geometries, like a ring magnet with a hollow shaft, or where the coaxial placement is otherwise not feasible.
For a while now moteus has had the ability to configure what action takes place upon a CAN message watchdog timeout during position control mode. If configured with 'servo.default_timeout_s' moteus requires that CAN messages be sent at a regular rate. If ever a message is delayed by more than the timeout period, the mode switches in a latching manner to the “timeout” mode, where a special action is undertaken. This is configured with ‘servo.timeout_mode‘ in tview, and as of firmware release 2023-07-25 the available values are as follows:
0: When set to 0, upon timeout, the driver will enter the equivalent of the “stopped” mode and all power will be removed from the motor phases. This will cause the motor to free spin as if there was no driver attached.
10: “hold position”. Here the controller will first decelerate to 0 speed, and then hold position as if a “d pos nan 0 nan” command were issued. Deceleration happens using the default configured acceleration limit in 'servo.default_accel_limit' and the default PID gains are used to hold that position.
12: “zero velocity”. Here the controller enters a mode where 0 speed is commanded with no constraint on position. It provides a damping effect on the motor to remove energy from the system. The default kd PID gain is used to control this effect and the configured kp and ki terms are ignored.
15: “brake”. For mode 15, the controller shorts all three motor phases together. This also provides a braking force. The amount of braking force is not configurable, but is also not reliant on firmware executing to operate so will function over a larger range of potential failure modes.
The perils of technical videos is that they can become out of date pretty quickly. For the moteus getting started guide, it has amazingly held up pretty well, but enough has changed since 2021 that it is time for a new one. Now we can use acceleration and velocity limited commands instead of the deprecated stop position and get nicer 4k b-roll for all the intermissions. Everything in the old video, and in this new one, is applicable to both the moteus-r4.11 and the moteus-n1.
Back last year, I walked through bisecting and debugging an annoying problem that caused the STM32G4 ADC on the moteus controller to exhibit higher than expected noise in result largely to either the exact placement in flash of the initialization code, or to the exact timing of the initialization. While the immediate glaring sharp edge was removed, the resulting performance still was confusing to me, and looked like it was not yet optimal. Further, a moderate percentage (2% or so), of production boards failed end-of-line tests related to the current sense noise in ways that were hard to fix by swapping components. Because of this, I wanted to dive in and investigate further. This is that process.
Tips and theories from the interwebs
In response to the original article, I received a *wide* variety of tips, leads, theories, and proclamations, some of which were more credible than others. Here are the top categories:
Incorrect clock or prescaler configuration
Some sort of clock domain synchronization problem causing prescaler configuration to be incorrectly set
Insufficient VREF+ decoupling
ADC initialization phasing
Some of these could be easily ruled out using just the information in the original article. Notably that the prescaler was documented to have been set to a frequency that was well within the range of what the ADC was capable of and that it was also read back from the peripheral and shown to have the correct value later in time.
That rules out the first two issues. To tackle the rest, I first needed to build some tools.
Quantifying the problem
Before attempting to confirm or refute the remaining hypotheses, I wanted to be able to better quantify the problem. In that vein, I made some tools and scripts which would attempt to render what noise was present as a function of reported ADC counts. The idea was to sweep each phase of the motor through a range of fixed voltages, and capture a time domain signal at a variety of steady state points. Then, we would plot the magnitude of the noise, and what the the frequency components looked like throughout the range.
The resulting tool set consists of a set of scripts to perform the sampling and then an interactive matplotlib based tool to render the results. It looks like this:
The top 3 plots in the top window show log-normalized spectrograms at each ADC count for each channel. The line graph in the top window shows the overall standard deviation of each channel at each ADC count. The bottom window shows the time domain signal at any given point — clicking on any of the top channels will switch to that given time domain plot. This lets you explore many possible hypotheses, especially those that are not necessarily best represented as noise in the frequency domain. Since the sweeps are over fixed voltages on each channel, the range of ADC values sampled are not the same for each channel, as the zero point bias is different for each channel and board. The big “X”s on the standard deviation plot shows where the zero point was for each channel.
In this particular plot (which is from a random intermediate test), you can see that there are definitely some frequencies which have noise that is periodic at every 8 LSB interval, channels 1 and 2 have significant wide-band noise from around 2030 to 2048, and channel 0 has an additional high frequency noise component from around 2041 to 2048.
Cycle accurate phasing
The next piece of infrastructure I needed to investigate this was to be able to make cycle accurate delays on the STM32G4, and have a delay end at a specific value of the global cycle counter. This is actually relatively hard, as there exists a flash accelerator, instruction cache, and data cache which all work to make accurate timing difficult, yet cannot be permanently disabled or things run too slowly. This is combined with the fact that when operating under the debugger, peripherals like the cycle counter are unreliable.
My solution was to first run the delay completely from the STM32G4’s CCM SRAM (which the ISR in moteus already was using). This takes the flash accelerator out of consideration. Next, the instruction and data caches are manually flushed before entering the critical section. Finally, the entire routine is implemented in assembly, with a superstitious number of alignment directives thrown in for good measure.
To calibrate the offset constants, I carefully set a breakpoint only on the first instruction after the sequence, and ensured that the CPU’s cycle counter was equal to the intended calculated one at every instance across a few different shufflings of code and compilations.
At this point, I could cause each ADC to be initialized at a specific CPU cycle counter value (or really any other operation), in a repeatable manner.
Preemptive solution: VREF+ decoupling
The ADC in the STM32G4 series operates on a successive approximation principal using switched capacitors. With that technique, at each clock cycle, various capacitors are either switched to VREF+ or to ground in order to more closely approximate the captured input voltage. Thus large current transients on VREF+ can occur, especially when the higher order bits in the sample are altered. These large transients, coupled through higher than desired impedance in the VREF+ signal path, can the VREF+ voltage to sag, resulting in faulty comparisons. The low-pin count STM32G4 variant that moteus uses is even more susceptible to this phenomenom as it has no dedicated analog ground pin. For the UQFN48 package, all grounds are tied to the exposed pad under the chip.
This particular failure mode is most likely to occur at voltages just below the halfway point, so below 2048 for the 12 bit STM32G4 ADC. If the first comparison is performed with a sagging VREF, and the actual voltage should have been, say 2044, it may erroneously be placed in the >= 2048 bucket, and then all subsequent comparisons will report it to be smaller thus resulting in exactly 2048 being the final result.
Even before attempting diagnosis, I went ahead and improved the situation for the moteus-n1, as it was in the design process anyway. The moteus r4.11 VREF+ decoupling did meet the official datasheet constraints of a 1uF ceramic capacitor close to the chip, but the ground path for that capacitor may not have been optimal, and 1uF may be insufficient for 5 simultaneous ADC operations. For the moteus-n1 r1.3 board, both a larger 4.7uF capacitor was used for the bulk, along with a separate smaller, 0.1uF capacitor. Also, additional ground vias were placed under the exposed pad along the entire side of VREF+, so that the full current path is short. Here is the resulting layout:
Here, pin 20 is VREF+, C10 is the primary 4.7uF decoupling capacitor, and C87 is the smaller capacitor. The ground path for the large capacitor is basically as short as physically possible, and 4 vias connect the ground plane to the STM32G4 all along that side.
Experimental flailing
Given these tools, I was able to make some progress.
First, I ran the before and after on the fix from the original post:
Problematic on left, “improved” on right
The fix definitely made things better, in that channel 0 and 2 had only a small non-linearity around 2048, although channel 1 still had a drastic deviation around 2048.
From looking at this, and fiddling with things, it seemed that with the moteus ADC configuration, the ADCs were definitely sensitive both to the exact cycle count phasing between their initialization, and the exact cycle phasing between any ADC and some system level phenomena. After a lot of fiddling around with the above tools, I hit upon the idea of sweeping through different possible options of inter-ADC cycle timing and from-boot cycle timing.
Note, for the purposes of the results below, the moteus controller was configured with a 170MHz AHB clock, a 2x divider to the APB1/APB2 clocks, the ADCs were configured to run asynchronously, their clock source was the peripheral clock, and the ADCs had an 8x prescaler.
Here are some example plots showing a few of the sample permutations:
8 LSB periodic noise that various with the CPU cycle modulo 2In this instance, ADC3/4 were initialized with a 4 CPU cycle offset from ADC1/2
No matter the spacing between ADC initializations, the results of all ADCs were materially better or worse depending upon when the first was enabled on a 2 CPU clock cycle boundary. I did not have a repeatable reference from power-on for the system, so it was arbitrary which was good or bad, but in any given firmware image it would look like:
first adc enable at CPU cycle offset 0: good
first adc enable at CPU cycle offset 1: bad
first adc enable at CPU cycle offset 2: good
first adc enable at CPU cycle offset 3: bad
…
Similarly, no matter what the state of the system from the first ADC enable phase, things could get worse if the individual ADCs were started in different phases relative to each other. This relationship was more complicated, but roughly followed a 16 CPU clock cycle. Results were best when the CPU cycle counts between when the ADCs were enabled was an even 16 cycles and was various degrees of worse at different phasings.
The “actual” problem
After figuring out the above, I was close to a fix. I could easily control the spacing between ADC initializations, but had not discovered a way to control the phasing between the first ADC initialization and whatever system level phenomena led the to the 2x period (likely due to the 2x AHB/APBx divider). It was at this point that I decided to go looking through the ST errata for the STM32G474 another time to see if I had missed anything.
And lo, while I hadn’t missed anything before, a new errata had been added in March 2023 which pointed to a more systematic resolution!
The errata gives three possible resolutions:
1: Do not execute ADC conversions concurrently. This is both not feasible in moteus, and in my experiments it does not completely resolve problems anyways.
2: Use the same clock for all ADCs AND an ADC prescaler of 1x. This is also not feasible for moteus, as a prescaler of 1x would have the ADC running much faster than the maximum possible speed.
3: Use a synchronous clock, the same clock configuration for all ADCs, and trigger them all simultaneously from a hardware timer with a compatible prescaler. This seems like a pointer to the best option, and hints at explanations for the problems I had before.
This errata perfectly describes all the confounding symptoms I initially had, where inserting various NOPs or re-arranging the code would cause the ADCs to either work well or give terrible results. Initializing the ADCs back to back made it much more likely that poor phasing would result, where doing them separately gave a larger room for chance to intervene.
Resolution
With the above errata and my experiments above, I now had a concrete path to a fix. The problem which had been hardest for me to resolve was the ambiguity about which of the APB phases the ADCs were enabled in. Using synchronous clocking presumably ensures that it is always the same one, and hopefully that phase is the “good” one.
For the triggering aspect, I tested two approaches, each of which gave identical ADC performance in my experiments. First, I switched the moteus software triggering to use a hand-rolled assembly block which triggers all the ADCs in 3 consecutive CPU cycles. Second, I had the PWM timer trigger a second timer through the internal hardware matrix, which then triggered the ADCs through the internal triggering mechanism. For whatever reason, TIM5 is incapable of driving the ADCs directly, thus the indirection requirement. This triggering improvement combined with the synchronous timer configuration removed all cases of periodic 8 LSB noise and instances of extremely large count 2048 noise.
With those changes in place, the results, while not perfect, are much improved. I verified them by using a variation of the cycle accurate delays above to add various delay phases of 1-8 cycles into the pre-main execution and between each ADC initialization to verify that the results were just as good in all cases. Interestingly, across a range of boards the moteus-n1 r1.3 results are about as good as the r4.11 results, which indicates that the VREF+ decoupling improvements, while a good thing overall, were not a significant contributor to at least these problems.
Final results, r4.11 left, n1 right
The git change implementing this can be found here:
As should be obvious, all of the above results above have one channel with significantly higher count 2048 noise. Interestingly, which ADC and channel it is differs between the r4.11 and n1. On r4.11 the offender is ADC1/IN12 (PB1) while on n1 it is ADC2/IN3 (PA6). I don’t know if it is channel specific within the MCU, or some board level deficiency that manages to only impact one channel or what. It still occurs even if the ADCs conversions are performed sequentially rather than in parallel, which at least rules out ADC cross contamination.
For now though, I’m leaving that as “future work”! Send me your thoughts!
If you look around online, therearelotsofexamplesofPCBtest fixtures used to perform end of line testing. In the low to medium volume scale, nearly all of these are either clamshell or 2 side affairs, where probe pogo pins or interfaces are connected to the bottom and top of the board.
When developing the moteus-n1, one of the challenges was the number of right angle board edge connectors it has. Those right angle connectors are what allow it to maintain a very low overall stack height when installed in applications, but are also much harder to perform testing on, since by definition the access points are not vertical. On the base n1, there are 6 total right angle connectors, 2 on each of 3 sides, and future variants may have additional bottom side CAN and power connectors populated to make 8 total right angle connectors.
In very low volume testing, say less than a hundred a day, it is feasible to just have a human manually connect cables to each of the connectors for each board being tested. As the volumes scale up though, this becomes a big bottleneck, and if the testing is not performed in a low-wage country, it can amount to a measurable portion of the overall product cost.
Design Overview
Thus, meet the test fixture I built for the moteus-n1:
It is manually actuated with a single big lever. As you pull the lever down, a series of guideways cause first the top plate to clamp the board into position vertically, and then subsequently three side plates slide in horizontally to attach probe points to three of the side edges. It is mounted in an 2020 extrusion frame that also hosts a 24V power supply, a Raspberry Pi and mjbots pi3hat, a lightly modified mjbots power_dist, and a custom STM32 nucleo I/O expander board.
I can’t say that the mechanics of the system are particularly elegant, but they work. When I started on the project some time ago, I went looking through a bunch of mechanisms to try and find something simple that could sequence the motions I wanted. I got pointed to thang010146’s youtube channel (for which you can get everything offline too) which is an amazing repository of ideas! However, after looking at a large fraction of those, and other resources to boot, the best I had come up with was a cam-like system where bearings were made to slide through pathways with a controlled profile. The profile can then be tailored to create arbitrary motion profiles for each of the driven pieces.
For instance, the vertical aspect looks like this in the closed position:
The guideway in the “closed” region forms an arc centered on the center of rotation of the handle, this keeps the top plate stationary. Then there is a region where the top plate is lifted, and finally a region at the end which is once again an arc about the center of rotation when the top plate is at its maximum extent. Thus, when the handle is pulled, the top plate initially remains level, drops down, and for the last N degrees of handle rotation, remains fixed at its lowest point.
The side plate that engages from the rear of the board is constructed nearly identically, with a guideway on the same face of the handle. The two plates that come from the left and right use a similar guideway, but here the guideway is in the curved section of the handle, which was harder to model in CAD.
This view from the back shows that the guideway for the left and right rails runs straight during the bulk of the handle pull, and then near the end pushes the side probes in towards the board. 3D printing this geometry was challenging, and I ended up printing a closed channel, and then post-processing it to remove a thin wall after printing was complete.
For the probes, each of the 5 sides has a removable probe plate which can be easily swapped out or changed. Most of the probes use the same modular test pogo pins that the most recent r4.11 test fixture used. Most of these pogo pin receptacles used the same installation approach that the previous fixture used, in that the holes for them were directly 3D printed. However, for the 1.5mm pitch JST ZH pins, PA50 size probes were required, and I could not get my prusa to reliably print usable hole sizes. So instead I had it print a tiny registration hole, and then used a $15 micro-PCB hand drill in two stages. First at 0.8mm all the way through, then at 0.9mm halfway through.
This allowed the receptacles to be pressed in reliably with a small arbor press:
This worked well for all the things that could be pogo-pinned, however the JST GH connectors are not amenable to be probed with any of the standard probe shapes. For those, I instead hand whittled mating GH connectors by removing the locking latch and shaping the remaining plastic to help them mate from a wider range of angles and offsets. Then I hot-glued them into place into the removable probe plate. I will admit they were finicky to get working well. The probe plate got re-printed a few times to line things up well, and the hand-whittling was an iterative process until I could get the connectors to reliably mate into the board.
Like in the previous fixture, a mj5208 motor is installed in the bottom, to provide a way to test the magnetic encoder and to test the power stage of the driver.
The I/O expander is an STM32 Nucleo board mounted into a simple custom PCB. It has a bunch of 0.1″ pin header connectors for all the probes to connect to, and routes all the things that need to be sensed with ADCs to resistor dividers, all the GPIOs to appropriate pins on the nucleo, and anything that is tested with an external device like the RS422 transceiver to yet other connectors. I populated it by hand, and included a 0 ohm resistor on nearly every line so that I could easily disconnect things that didn’t work, which ended up being necessary when I made some mistakes in pin assignment.
The Raspberry Pi 4 that runs the software has a mjbots pi3hat attached. That powers the rpi from the 24V input and also is used for a CAN connection to the power_dist board. A quick 3D printed mount holds it together and lets it clip onto the frame.
A hall sensor is mounted on the spring clip, so that the test application can automatically start the test cycle when the handle is fully closed.
Operation and Software
The software used is a derivative of the software used for the previous test fixture. First, it was updated to use the I/O expander board to take all the new measurements, such as verifying that all the ground and power pins on each connector are working properly, as well as all the additional GPIOs. Second, and more obvious, is that it was updated to use the python textual library to create a two-paned application.
One pane shows the ongoing results of the current board being tested, and the other pane shows the previous board’s results. The script was also updated to play an audible sound when a board is ready to be removed, so that uploading results from the previous board can take place in parallel with the beginning of the next test cycle.
Conclusions and Future Work
With this fixture, it is possible to reliably test and package more than 70 boards an hour, including all the additional test coverage. That is up from around 30-40 boards per hour with the old fixture. While the r4.11 has less need for side probing, it is still possible that a fixture based on this designed for it will be made at some point, if nothing else to add verification of the ABS port and the improved test cycle time.
I don’t normally post about moteus firmware releases, but there have been many in the last few months that fix some long standing bugs and regressions. First, the current release as of this post is:
It is embarrassing how long this issue has been outstanding, but ever since the flexible I/O system was introduced, there has been an issue where if the gear reduction was configured to a value other than 1, then the output position would drift from the source encoder over time. If control was not active, you would see the position drift, and if control was active, then moteus would report the correct position, but the actual position would drift.
Non-zero goal velocities with limits
Last year, when acceleration and velocity limits were implemented, the semantics of a command with a non-zero velocity was that the controller would try to reach the exact position as stated in the command while traveling at that velocity. If the velocity was non-zero, that often meant that the controller would need to back up a lot, then re-accelerate. Needless to say this is not the most useful semantics.
Now, moteus treats a command with non-zero velocity as “follow the trajectory x = x_command + v_command * t”. This is what most users expected and is more useful.
Bootloader operation in degraded CAN environments
The primary moteus firmware has long had two mitigations for degraded CAN environments. First, if a query message has the BRS (bit rate switch) flag disabled, then the controller will respond in kind. This allows the host to force operation at 1Mbps instead of the normal 5Mbps. Second, moteus will automatically attempt to rejoin the CAN bus if a busoff event occurs, say because some device was connected with an incorrect bitrate.
Those were helpful, but the bootloader lacked those fixes, so flashing new firmware in a degraded CAN environment was either unreliable, or not possible at all. No longer!
Misc
That wasn’t even all, a number of smaller fixes were made:
It used to be possible to trigger a firmware assert if position mode was commanded within a few ms of power on.
The “conf default” command would reset the configuration, but those new values would only actually be used for a subset without further action.
It was possible for the initial position to be outside the range of -0.5 / +0.5.
The ADC performance variability with firmware versions is now hopefully resolved for good (to be detailed in a later post).
Thanks for all your feedback to help find and fix issues!
This variant is intended to be more feature-full, higher performance (and higher cost). Here are some bullet points of the biggest differentiators with r4.11:
moteus n1
moteus r4.11
Price
$159 USD @ qty 1
$104 USD @ qty 1
Size
As small as 46mmx46mmx8mm with optional back connectors omitted.
58% of the volume, 87% of the top down footprint size of r4.11
53x46x12mm
External Peripherals
Each of the auxiliary connectors supports SPI, UART, ADC, SW & HW Quadrature, Hall sensors, and I2C.
5V and 3.3V is provided on each connector to power peripherals (100mA for each voltage available combined between both connectors).
I2C pullups are configurable on each connector.
All 4 pins on AUX2 are 5VT, the two non-SPI pins on AUX1 are 5VT.
ENC (AUX1) supports SPI, Hall and ADC.
ABS (AUX2) supports UART and I2C and are both 5VT.
RS422
Built in RS422 transceiver for communicating with RS422 / BiSS-C encoders (BiSS-C / SSI not yet supported in software).
None
Voltage
8-54V, 48V nominal
8-44V
Peak Output
100A output phase current, 1200W
100A output phase current, 500W
Continuous Output
9A output phase current ambient, 18A w/ thermal management
11A output phase current ambient, 22A w/ thermal management
CAN fault tolerance
58V bus fault tolerance
12V bus fault tolerance
Power Connectors
Solder pads for DC bus input, one always present XT30, one optional XT30
2x XT30
The short form is that the n1 does not really expand the set of motors that can be usefully driven, but does enable operation in 48V systems, is more compact and electrically robust and provides significantly improved external peripheral support.
Don’t worry, the existing r4.11 controllers aren’t going anywhere and are still being produced. The moteus-n1 series just complements them by enabling new applications.
Firmware
Both the moteus n1 and r4.11 share the same open source firmware, although the n1 requires a newer release to operate. Command, control, and monitoring work identically on both, including tview, the python library, and all CAN formats. The only exception is that the n1 has more external pin configurations possible as shown in the reference documentation’s pin capability table.
Accessories
The mechanical form factor of the n1 is different from r4.11, and thus the existing devkit bracket and heat spreader can not be used. New variants exist for both of those that are compatible with the n1.
Finally, all the external peripheral connectors use JST-GH connectors, so we now have headers and pre-crimped wires available for the JST GH6 (used for the RS422), JST GH7 (used for AUX2) and JST GH8 (used for AUX1).
Starting today we’re making some moteus-n1 units available in a paid beta program, with errata documented here. Up to 2 controllers in either devkit or bare board form (both bare board and devkits variants count towards the total) can be ordered per customer via the links above. There is no technological limit if you try to exceed that, you will just have your order cancelled with a note to try again with fewer.
A Modicum of Fun 