Dynamixel 2.0 servo side: implementing Fast Sync/Bulk Read

This post documents what I learned implementing Dynamixel Protocol 2.0 on the servo side. OpenServoCore has since moved to its own wire protocol , but everything below stands on its own if you are building a Dynamixel-compatible device.

Fast Sync Read and Fast Bulk Read are the instructions that make a Dynamixel 2.0 bus fast. You send one request and get one shared reply carrying every servo’s data back-to-back. They are also, as far as I can tell, completely undocumented from the servo side. There is plenty written about sending a Fast Sync Read, but nothing about being one of the servos answering it.

This is part 2 of a pair. Part 1 covered how a servo detects the moment a packet ends on the wire, and I’ll assume that vocabulary here (wire-end, publish time, jitter, RDT). This article is the build itself. It covers the wire layout, the CRC math, the slot arithmetic, the hardware transmit start, and the CRC patch that races your own transmit DMA, each with the code. The worked example is the CH32V006 ($0.16, 48 MHz, no crystal), and the code comes from the frozen DXL-era tree , trimmed to teaching size. Every mechanism has an equivalent on any small MCU with a timer, DMA, and a UART. If you are new here, the project overview has the full picture.

TL;DR

Here is the whole recipe up front:

• The per-block CRCs are cumulative checkpoints. Seed your CRC state from the two bytes your predecessor just transmitted, extend it over your own block, and patch it into your reply. You never checksum anyone else’s bytes.
• Anchor your slot with one clock reading, back-projected through the DMA byte count, no per-byte timestamping hardware needed.
• Fire the transmit with hardware. A timer compare triggers a DMA transfer that enables the TX DMA channel, so there is zero CPU in the deadline path.
• Same rule as part 1, aim late and never early, except now the budget is one byte-time.

Still suggest reading through though. The details are where all my bench time went.

What Fast Actually Is

A normal Sync Read asks N servos for data and gets N separate reply packets, each with its own header, its own reply-delay gap, its own overhead. The Fast variants (Fast Sync Read, instruction 0x8A, and Fast Bulk Read, 0x9A ) collapse all of that into one shared status packet.

The first servo transmits the packet header plus its own block. Each following servo appends its block directly onto the wire, ideally with no gap at all, so the host’s UART sees one continuous packet.

Let’s get concrete about the bytes, because this layout drives every formula below. The first servo (slot 0) transmits:

FF FF FD 00   header
ID            the responding chain's packet ID
LEN LEN       packet length, 2 bytes
INST          0x55, status
ERR           slot 0's error byte
ID0           slot 0's servo ID
DATA...       slot 0's data, L bytes
CRC CRC       cumulative CRC so far

That is 10 bytes of overhead plus L data bytes plus 2 CRC bytes. Every later servo (slot k > 0) transmits only:

ERR           this slot's error byte
IDk           this slot's servo ID
DATA...       this slot's data, L bytes
CRC CRC       cumulative CRC so far

That’s 2 + L + 2 bytes, with no header, length field, or instruction byte. With 2 data bytes per servo, slot 0 puts 14 bytes on the wire and every successor puts 6.

Those two trailing CRC bytes per block are what makes the whole thing work, so they get their own section.

The Per-Block CRC Checkpoints

Each block ends with a CRC, and that CRC is cumulative. It is the running checksum of everything transmitted so far, meaning the header, every earlier servo’s block, and yours. Each servo picks up the checksum state from the two CRC bytes its predecessor just put on the wire, continues it over its own block, and writes the updated state as its own trailing CRC. The e-manual says it in one easy-to-skim sentence. “CRC values are used for internal calculation in DYNAMIXEL to confirm packet integrity between DYNAMIXELs.”

The reason resuming works at all comes down to two properties of the specific CRC Dynamixel uses (CRC-16/UMTS). Its initial value is 0x0000 and it applies no final XOR, so the running state at any point IS the wire value at that prefix. Store the two bytes, and you’ve stored the state. The whole resumable-checkpoint mechanism is then one function:

/// Extend a running CRC-16 over more bytes and return the new state.
/// Seed with 0 for a fresh packet, or with the two checkpoint bytes
/// carried from earlier bytes of the same packet.
pub fn crc16_continue(state: u16, bytes: &[u8]) -> u16 {
    let mut crc = state;
    for &b in bytes {
        crc = (crc << 8) ^ TABLE[(((crc >> 8) as u8) ^ b) as usize];
    }
    crc
}

You can unit-test the property in one line. Folding "1234" and then continuing over "56789" gives the same result as folding "123456789" from zero. The shipped code keeps exactly that test next to the table. A bonus of the same property is that the constant FF FF FD 00 header prefix can be pre-folded into a compile-time constant, so a fresh packet’s CRC never starts from scratch either.

This design is actually brilliant. The final block’s CRC ends up mathematically identical to a CRC over the entire packet, so the host gets whole-packet integrity, yet no servo ever checksums anything but its own bytes. The work stays proportional to your own reply no matter how long the chain is.

How I Misread The Spec

A warning about reading the spec, because I got this badly wrong. It is very easy to skim the packet diagrams and conclude there is one CRC at the very end. That would mean the last servo has to checksum the entire packet, hundreds of bytes it never sent and can only observe by listening on the wire.

I implemented exactly that misreading. I built three generations of machinery to serve it. Then I re-read the sentence quoted above and deleted all of it.

The embarrassing part is how long the wrong model survived, it lasted weeks. And the reason is that I had written both ends of the bus myself, so my host implementation happily agreed with my servo implementation. So make sure you test against the incumbent early. And if your Fast implementation ever seems to require heroics, go re-read the packet layout, because the checkpoint sitting two bytes behind you already contains everything you need.

What A Servo In Slot k Must Do

1. Parse the request, find your ID at position k, and compute how many wire bytes precede your block.
2. Encode your reply early. Error byte, ID, data, and two placeholder bytes where your CRC will go. You can’t know the CRC yet.
3. Anchor on the shared packet’s observed start. Slot 0 replies like a unicast read, at wire-end plus RDT. That is the only RDT that matters, later slots ignore their own. Everyone else times their turn by observation, because per part 1, firing on arithmetic alone risks the one failure you can never allow, transmitting early into your predecessor.
4. Start transmitting within the hand-off budget, about one byte-time (3.33 µs at 3 Mbaud) past your predecessor’s last bit for a zero-gap chain. Arriving late only costs a tolerated gap, but arriving early corrupts the shared packet.
5. Read the predecessor’s checkpoint, finish your CRC, and patch it into your reply, all while the reply is already leaving the wire.

Steps 3 to 5 are where all the engineering lives. Let’s go through them, code first this time.

One thing to know up front is that the final architecture is asymmetric on purpose. Transmit timing is hardware and receive timing is software, because TX jitter lands on the wire as your start bit while RX jitter only feeds arithmetic with tens of microseconds of tolerance. Part 1 walks through why, and this article is where it pays off.

The Slot Math

The offsets follow directly from the wire layout above (fast_shape.rs ):

const SLOT0_OVERHEAD: u32 = 10; // FF FF FD 00 + ID + LEN(2) + INST + ERR + id0
const SLOT_OVERHEAD: u32 = 2;   // ERR + idk
const CRC_BYTES: u32 = 2;

fn first_block_bytes(l: u32) -> u32 { SLOT0_OVERHEAD + l + CRC_BYTES }
fn successor_block_bytes(l: u32) -> u32 { SLOT_OVERHEAD + l + CRC_BYTES }

/// Wire bytes preceding slot k's block, uniform data length l per servo.
fn bytes_before(k: u32, l: u32) -> u32 {
    if k == 0 {
        0
    } else {
        first_block_bytes(l) + (k - 1) * successor_block_bytes(l)
    }
}

With 2 data bytes per servo, bytes_before gives 0, 14, 20, 26, and so on. For Fast Bulk Read the per-slot lengths differ, so you accumulate the sum while walking the request’s slot list instead of multiplying. It is the same idea either way. The request tells you every listed servo’s data length, so the whole schedule is known the moment the request parses.

Multiply bytes_before(k) by the byte-time and you have your slot’s offset from the shared packet’s start, in clock ticks. But how do you actually measure the start of the shared packet?

Anchoring The Slot

When the Fast request parses, a successor slot stages its reply and parks, recording the ring position where the shared status packet’s first byte will land. It then opens a temporary per-byte receive-interrupt window (the RXNE window from part 1’s gotcha list). These are the only per-byte interrupts in the whole design, and the window closes as soon as the status packet’s first byte shows up.

At that first wake, the servo reads its free-running clock once and back-projects to the packet’s start using the byte count the receive DMA has already published:

/// One clock reading, projected back to the shared packet's first byte.
pub fn status_start_tick(&mut self, now: u32, byte_ticks: u32,
                         start_cursor: u32) -> Option<u32> {
    self.ring.publish(self.rx_dma.remaining());     // NDTR -> published count
    let published = self.bytes_consumed + self.ring.available();
    let n = published.wrapping_sub(start_cursor) as i32;
    if n <= 0 {
        return None;                                // first byte not here yet
    }
    Some(now.wrapping_sub(byte_ticks * (n as u32))) // back-project n bytes
}

If the wake happened three bytes into the packet, the start was three byte-times ago. The projection has the self-correcting property from part 1. The clock reading and the byte counter advance together, so interrupt-service lag cancels out of the difference. Better still, lag can only make the wake later, never earlier, so the derived start can slip late but can never fire you early into your predecessor. This safety comes from the structure itself, I did not have to tune anything to get it.

The fire deadline is then one multiply and one add:

fire_deadline = status_start + bytes_before(k) * byte_ticks   // no RDT term

There is deliberately no RDT term for successors. RDT applies to single-target replies only, and the spec’s chain examples bear that out. Slot 0 is the exception, it fires like a normal reply at wire-end + RDT using part 1’s machinery.

On the bench, successor slots land at −0.29 µs median with 0.125 µs of scatter at 3 Mbaud. That is an order of magnitude inside the one-byte-time budget. Not bad for a software anchor!

Firing With Hardware

A software-fired transmit start costs interrupt entry plus a jitter tail. I measured +1.22 µs at p99.9 and +2.85 µs max. That is over half a byte-time at 3 Mbaud, landing directly on the wire. The shipping design removes the CPU from the moment entirely by chaining three hardware behaviors. All three are verified on the V006 bench (tx_kickoff.rs ), so look for equivalents on your silicon.

Piece 1. A timer compare with no pin. TIM2 channel 4 runs as an output compare in “frozen” mode, output disabled. The compare match still fires its DMA request, so you get a precise hardware event without spending a pin:

fn ch4_oc_frozen() {
    TIM2.ccer().modify(|w| w.set_cce(CH4, false));  // no pin drive
    TIM2.chctlr().modify(|w| {
        w.set_ccs(CH4, Ccs::OUTPUT);
        w.set_ocm(CH4, Ocm::FROZEN);                // match fires, pin untouched
    });
}

Piece 2. A DMA channel whose destination is another DMA channel’s control register. This is the strange one, so read it twice. The UART’s transmit DMA channel (CH4 on this chip) is fully configured ahead of time, buffer, length, everything, except its enable bit is off. A second channel (CH7) is armed with a single 32-bit transfer whose source is a precomputed word (CH4’s config with the enable bit set) and whose destination is CH4’s own control register. The timer compare triggers CH7, CH7 writes “go” into CH4, and the first byte leaves, with no interrupt anywhere in the path:

pub fn arm(compare: u16) {
    timer::set_ccr4(compare);            // the deadline, low 16 bits
    timer::clear_cc4_flag();             // a stale match must not count
    dma::disable(Channel::CH7);
    KICKOFF_WORD = ch4_config_with_enable_set();
    dma::configure(Channel::CH7,
        /* dst */ DMA1_CH4_CR_ADDR,      // another channel's control register
        /* src */ &KICKOFF_WORD,
        /* n   */ 1);                    // one 32-bit transfer
    dma::clear_tc_flag(Channel::CH7);
    dma::enable(Channel::CH7);
    timer::set_cc4_dma_request(true);    // open the request gate LAST
}

Piece 3. The bus-direction pin rides a second compare channel. TIM2 channel 2 drives the TX-enable pin in “active on match” mode with the same deadline, so the transceiver turns around exactly at the fire moment, again with no ISR. The pin gets dropped later by the transmit-complete interrupt, where timing no longer matters.

Scheduling into a 16-bit timer from a 32-bit clock is its own small trick. Zero TIM2’s counter and the 32-bit SysTick together at boot and never reset either again (this is what part 1’s “never reset the clock” rule was for). The timer is then simply the low 16 bits of your clock, and a 32-bit deadline truncates directly into the compare register with deadline as u16.

Now let’s go through the three guards. Each one of them came from a real failure on the bench.

Guard 1. Set and recheck. After arming, re-read the counter. If (compare - counter) & 0xFFFF exceeds the largest legitimate future, the moment already passed while you were arming. Force the direction pin active and re-aim the compare just ahead of the counter so a real match fires it. You must re-aim on the V006, because the software-generated-compare-event bit (the SWEVGR register’s CC4G) is dead silicon. The write does nothing. Make sure you verify yours before depending on it.

Guard 2. Requests latch into disabled channels. On this silicon, a compare match that pulses a DMA request while the kickoff channel is parked delivers that request the instant the channel re-enables. I probe-verified this with a stale boot-time match that fired the kickoff about 720 ticks early and streamed two bytes onto the wire before the direction pin was up. Clearing the compare flag does not purge the latch. The gate has to be the request enable itself. Hold the timer’s compare-to-DMA request bit off while parked, and during arming, turn it on last, after the compare points strictly ahead and the flag is wiped. That’s the last line of arm() above.

Guard 3. Flag-gate the restore interrupt. The kickoff channel’s transfer-complete handler parks the channel again. Gate its body on the actual transfer-complete flag, so a stale pended interrupt can’t disable a freshly re-armed channel mid-window:

pub fn on_kickoff_complete() {
    if !dma::is_tc_flag(Channel::CH7) {
        return;                          // stale pend, not a real completion
    }
    timer::set_cc4_dma_request(false);   // request line dead while parked
    dma::clear_tc_flag(Channel::CH7);
    dma::disable(Channel::CH7);
}

There are two calibration notes to close the section. I had reserved a floor constant to back-date the compare, expecting to compensate for the silicon’s request-to-wire delay. When I measured it, the raw path already lands the first bit about 0.6 µs after the direction pin asserts, which is exactly the small positive lead you want, so the constant shipped at zero. Back-dating by even 8 ticks pushed the tail negative, meaning first bits leaving before the direction pin was up. That silently clips replies that begin with 0x00. And guess what a Fast block with a zero error byte begins with… yes, the common case. So make sure you calibrate against the measured wire-excess distribution, and treat a dropped leading byte as the failure signal.

Measured end to end, single-servo turnaround improved from about 151 µs with the interrupt-fired design to about 101 µs with the hardware kickoff.

The Checkpoint Pickup

Now let’s deal with the CRC. The staged reply is already streaming. The hardware fired it regardless of what the CPU is doing, and its last two bytes are placeholders that the transmit DMA’s read cursor is marching toward. The pickup must beat the cursor there.

The state machine has three phases:

DRAIN  while the predecessor window is longer than half the receive
       ring, take cheap intermediate wakes, one ring-bookkeeping pass
       each, so the published position can't alias before the pickup
WAKE   one timer wake, deliberately early-biased, scheduled at
       window_end - wake_lead (500 ticks, sized to worst-case
       interrupt entry, not typical)
loop:  checkpoint bytes in the ring?
           -> seed, fold, patch. done
       TX read cursor reached the CRC slot (its NDTR <= 2)?
           -> give up. ship the placeholder, bump a counter
       predecessor deadline passed with no checkpoint?
           -> re-arm one byte-time later and retry

The pickup itself reads the predecessor’s last two bytes straight out of the receive ring by producer offset, then runs the fold (fold_engine.rs ):

fn finalize_from_checkpoint(&mut self, checkpoint: [u8; 2],
                            sink: &mut impl CrcPatchSink) {
    let mut state = u16::from_le_bytes(checkpoint);   // seed: predecessor's CRC
    state = crc16_continue(state, &checkpoint);       // its 2 bytes are also data
    state = crc16_continue(state, sink.own_reply_bytes()); // your block
    sink.patch_crc(state);                            // overwrite the placeholder
}

Look at the second line carefully, because it is the detail most likely to produce an off-by-two implementation that only fails against real hosts. The checkpoint’s own two bytes are part of your covered span. You seed from them as state AND fold them in as data, because they are stream bytes for your block’s CRC.

The rest of the details that make it correct, each one verified in the shipped code:

• The checkpoint is inherited untrusted, on purpose. There is no validation of the predecessor’s block. Per-block validation is the host’s job, and a corrupted upstream block fails the host’s chain check with or without your patch. Validating would cost a CRC pass per snooped block and buy nothing.
• Bias the wake early. Waking early only costs a few polling spins, each a fraction of a byte-time, but waking late eats the only margin that matters.
• A silent predecessor degrades correctly. The hardware fire already happened, so the reply ships with its placeholder, a diagnostic counter increments, and the host sees a bad block. I want a broken chain to look broken on the host side, instead of hiding the damage.
• Short windows run inline. At high baud, a short predecessor can be fully in the ring before any wake is worth dispatching, so when the deadline is within about 50 µs the pickup runs right at observation time instead of scheduling a wake. A valid CRC on a slightly late fire is still better than a placeholder shipped on time.

Keep The Hot Path In RAM

This one deserves its own section because it cost me a full day and the fix is invisible in source code. Flash wait states on this class of chip cost more than the whole patch margin, so the pickup loop must execute from RAM. In Rust that is a link-section attribute:

#[cfg_attr(target_arch = "riscv32", unsafe(link_section = ".highcode"))]
#[inline(never)]   // keep the body out of flash-resident callers
fn checkpoint_pickup_body(/* ... */) -> bool {
    // publish ring, poll for checkpoint, fold, patch
}

Both attributes matter. The section attribute places the function in RAM (.highcode is this chip’s runtime-provided RAM-code section, your vendor will have an equivalent). The inline(never) keeps the compiler from inlining the body back into a flash-resident caller, which would quietly undo the placement.

And here is the gotcha that made this worth a section. Closures silently lose your section attribute, because each closure is its own unsectioned symbol. My pickup body started life as a closure, the CRC helpers inlined into it, the whole per-iteration loop landed back in flash, and the patch lost its race against the TX DMA. One stray flash-resident function measured as exactly the difference between 50-out-of-50 CRC failures and 0-out-of-50 at 3 Mbaud. Make the pickup body a named function, and check the final ELF with nm or objdump instead of assuming.

The final margin at 3 Mbaud came out to about a quarter of a byte-time, and the 1-byte-reply corner that broke every earlier design now runs 50-out-of-50 clean.

What Not To Build

Each of these was really built and really measured before it got deleted. Learn from my wasted bench time.

• Checksumming the whole predecessor window. This is the phantom problem from my spec misreading. I went three generations deep (interrupt-per-byte folding, scheduled catch-up folding, a wall-clock-anchored busy-wait fold), and each one worked in the middle of the test matrix and broke at a corner. The per-byte variant drove reply timing bimodal, ±45 µs. Under the correct model, none of it needs to exist.
• Per-byte interrupt work near a transmit deadline. That is 300k interrupts per second at 3 Mbaud, queueing behind and ahead of your own start-timing until the jitter goes multi-modal. The shipping design’s only per-byte window is brief, bounded, and never overlaps a deadline.
• Per-byte hardware timestamping of the receive side. I built it, measured it at about half the CPU, and deleted it. Part 1 has the full design and the verdict.
• Over-compensating the hardware fire. A “safety” lead constant can push your start early, and early is the failure mode. In my case, zero compensation measured on the bench beat any cleverness I could come up with.

The Clock Prerequisite

Everything in the slot anchor multiplies bytes_before × byte_time on your own clock. A crystalless chip’s oscillator is specified at ±1%, and real parts wander further. At a 128-byte offset, that is more than a byte-time of error, and your hand-off budget is gone before you even start. So a crystalless servo on a Fast chain must discipline its clock against the bus.

How OpenServoCore does that today, measuring the host’s crystal through the wire itself with no extra hardware, is its own article. See HSI trim: calibrating a crystalless MCU over the bus .

The Numbers

Measured on the CH32V006, at the end of the Dynamixel era:

The complete implementation and the bench measurements behind every number are preserved at the dxl-2-frozen tag of the OpenServoCore repo. Same disclaimer as part 1 applies here. The design notes in that tree are unreviewed AI-drafted working notes. The code is the part I actually stand behind, so skip the write-ups if you’re allergic to AI. And the code excerpts above are trimmed for teaching, so when they disagree with the tree, the tree wins.

What’s Next

Implemented correctly, Fast is a genuinely well-designed feature on a demanding wire format. But the demanding part is baked into the protocol itself. The format gives a servo no delimiter it can trust, so headers must be hunted and payloads must be stuffed. Throughput depends on microsecond hand-offs. And every timing tolerance turns into a clock-quality problem on chips that don’t have clocks. Those costs come with the protocol itself, and no implementation can avoid them.

This project’s next step was designing a wire format where those costs can be deleted. That story is in the protocol redesign article .