Hello, I’m working on getting my serpent robot operational. I’ve been doing some experiments with the LSS HT servos, and their torque is great. I’m seeing a strange issue, though: when I issue move commands to the servos and those look like they’d be high-torque moves, the servos seem to shut down for ~5 seconds. I don’t see any lights flash on the servos, and although I monitor the servo status every few seconds, I don’t see any servos getting stuck in e.g. Safe Mode, and I don’t see the start-up LED sequence, anyway. The only non-default configuration I’m using is changing the origin offset & setting the angular range to 1800.
I added event counters to the power-related subsystems in the robot, and I’ve got the telemetry flowing from the microcontrollers that communicate with the servos over a CAN-FD bus into Prometheus. I am 95% sure that the servos don’t look to be resetting, and the rest of the systems also seem to be operating without any issues.
Finally, I also noticed that the shutoffs are much more frequent when I’m communicating with the servos at ~600 commands per second (2/3 query commands, 1/3 position commands) at the default UART speed.
Can you tell us which firmware version they have? We might switch you to something experimental.
Also, if you’re using EM1 mode, which is intended to handle the move, you should switch to EM0.
I implemented a simple command in the can bus so that I can remotely communicate directly with the servos, instead of through the robot’s higher-level APIs. I’ve confirmed all 18 of the servos are on firmware version 368.
Switching to EM0 certainly makes the moves much, much faster. However, I still see servos doing their “limp for the next 5 seconds” thing when I have them make “big” movements (big meaning more distance/torque/etc required).
I’d be happy to try other firmware versions, and I’ll also add more diagnostics to alert if communications stop (in case I’m seeing “no update” instead of limp, and misinterpreting the results). I can also post a video of the behavior if that’s helpful.
edit: I added staleness tracking for the statuses I poll from the servos at 100Hz (2 servos per uart, querying temp, position, current, voltage, and status). I’ve confirmed now that the servos are continuing to report all of their information/stats as normal–could this be a behavior of a motor driver IC? I’ll also increase the rate I poll the servo status to see if briefly flashes to an unexpected status & back.
edit2: I increased the rate at which I poll the servos to make sure. This has revealed that the servos go to the “outside limits” state sometimes, but it seems to flash only briefly, so I don’t yet know the actual frequency.
Although the issue you are encountering is new to us, this may indirectly resolve it since most of the update involves aspects of the communication. You also might appreciate the Intermediate Position Management System (IPMS) in EM0 mode.
Hi @cbenson, I changed all the servos in the robot to v370. I still see the apparent 5s timeout pretty often when the servos are trying to move. I’ve attached a video that I took, so that you can see a few things: the servos have plenty of torque to move the robot (when they engage), but also they stop activating and come back after a bit (and I’m they’re still responding to commands, and they’re happy to report that they’re limp when they do stop).
The intended movement I’m doing is switching between 20º and 0º angle between each segment, so switching between straight & curled every 1 second. This is definitely an intense movement, but it reliably triggers the issue, the servos have enough torque to easily move this way, and the batteries can supply 10A at peak moves, so I don’t think it’s a brownout either (plus, it stays in EM0 mode, which I didn’t persist to EEPROM).
The video: LSS HT unexpected limps during high power travels - YouTube. I looked at the metrics during the video, and it looks like the worst case voltage sag sampled at ~5Hz was to 10.8V (independent 3s lipo on each segment). The highest current I saw was 1A. There were many transitions to Outside Limits and Limp statuses.
If you have all 14 HT1s moving at a time, they might consume up to 1.2A, so 16.8A total for a sudden move. It might brown out only momentarily. Is there any way to test this with a power supply which can provide higher current (like a LiPo pack or powerful wall adapter?). This is not an issue we have encountered before. Also, given the rate of movement, can you reduce the number of commands being sent and see if that makes a difference?
Each pair of HT1s is attached to a separate 3S Lipo pack that’s rated for 25A burst with a 10A continuous pack protector, and the remaining electronics consume around 150mA (additionally, each pair of servos has an independent serial bus & sercom unit on a separate microcontroller). Because of this, I’m pretty sure we’re not browning out. Additionally, I don’t think the servos are browning out because I don’t see the LED blink the “power on/bootup” sequence. Finally, in the video, the servos are getting only a single command every 1 second (although the query commands are continuing, the move commands are being sent at 1Hz).
Do you think I should try to reduce the query command rate and see if that impacts anything?
Indeed, you can see if it’s a communication issue (ex. buffer overrun) by decreasing the number of commands which are sent to each servo, as well as queries.
