Support programming of Arduino devices in serial passthrough mode (#5…

…129)

* Support DTR in serial passthrough mode to enable programming of Arduino
based devices such as MinimOSD.

Use 'serialpassthrough 5 57600 rxtx 56' and then use Ardino to program MinimOSD
Use 'serialpassthrough 5 115200' and then use MWOSD configurator to setup

* Fix comment for CDC_SetCtrlLineStateCb routine

* Handle F7 CDC interface

* Use strToPin() to allow easy port/pin specification

* Fix use of CDC_SetCtrlLineStateCb for all processor types

* Only set baud when specified

* Fix unit tests for cli

* Only register callback if needed

* Fix white space

* Provide implementation of IOConfigGPIO in SITL

* Update serialpassthrough help text

* DTR handling through serial drivers

* Fix F3, F7 and SITL builds

* If serialpassthrough command specifies baud rate of 0, set baud rate over USB. MWOSD configurator can now access config and reflash MinimOSD without rebooting and changing baud rate.

* Fix F3 build

* Fix failing unit tests

* Use resources to declare DTR pin assignment

* Don't assert DTR during normal operation as MW_OSD doesn't like it

* MW_OSD must be built with MAX_SOFTRESET defined in order to support DTR resets

* Minimise changes after dropping DTR pin param from serialpassthrough cmd

* Remove DTR pin param from serialpassthrough cmd

* Treat ioDtrTag as boolean in conditional statements

* Tidy buffer check

* Check buffer size in CDC_Itf_Control

* Fix unit test

* Add documentation for DTR

* Add note on MAX_SOFTRESET to documentation

* Remove superfluous function definitions

* Fix tabs

* Fix tabs

* Removed superfluous entried from vtable

* Backout whitespace changes unrelated to this PR

* Pass true/false to IOWrite()

* Fix line coding packing

* Add LINE_CODING structure defintion

* Revise serial documentation

* Prevent tx buffer overflow in serialPassthrough()

* Revert change unrelated to PR

* Review feedback from ledvinap

* Fix unit test

* Use PINIO to drive DTR

* Fix unit test

* Remove change unrelated to PR

* Fix SITL build

* Use shifted bits for mask definition

* Fix serialpassthrough documentation

* Only compile PINIO functionality if USE_PINIO defined

* IOConfigGPIO not needed

* Move cbCtrlLine callback to cli.c

* serialPassthrough params changed

* Check packed structure size

* Fix unit test

* Tidy up baud rate handling

docs/Serial.md

@@ -87,16 +87,63 @@ The allowable baud rates are as follows:
 
 Cleanflight can enter a special passthrough mode whereby it passes serial data through to a device connected to a UART/SoftSerial port. This is useful to change the configuration of a Cleanflight peripheral such as an OSD, bluetooth dongle, serial RX etc.
 
-To initiate passthrough mode, use the CLI command `serialpassthrough` This command takes three arguments.
+To initiate passthrough mode, use the CLI command `serialpassthrough` This command takes four arguments.
 
- serialpassthrough <id> [baud] [mode]
+ serialpassthrough <id> [baud] [mode] [DTR PINIO]
 
-ID is the internal identifier of the serial port from Cleanflight source code (see serialPortIdentifier_e in the source). For instance UART1-UART4 are 0-3 and SoftSerial1/SoftSerial2 are 30/31 respectively. Baud is the desired baud rate, and mode is a combination of the keywords rx and tx (rxtx is full duplex). The baud and mode parameters can be used to override the configured values for the specified port.
+ID is the internal identifier of the serial port from Cleanflight source code (see serialPortIdentifier_e in the source). For instance UART1-UART4 are 0-3 and SoftSerial1/SoftSerial2 are 30/31 respectively. Baud is the desired baud rate, and mode is a combination of the keywords rx and tx (rxtx is full duplex). The baud and mode parameters can be used to override the configured values for the specified port. `DTR PINIO` identifies the PINIO resource which is optionally connected to a DTR line of the attached device.
 
 For example. If you have your MWOSD connected to UART 2, you could enable communicaton to this device using the following command. This command does not specify the baud rate or mode, using the one configured for the port (see above).
 
  serialpassthrough 1
 
+If a baud rate is not specified, or is set to 0, then `serialpassthrough` supports changing of the baud rate over USB. This allows tools such as the MWOSD GUI to dynamically set the baud rate to, for example 57600 for reflashing the MWOSD firmware and then 115200 for adjusting settings without having to powercycle your flight control board between the two.
+
 _To use a tool such as the MWOSD GUI, it is necessary to disconnect or exit Cleanflight configurator._
 
-**To exit serial passthrough mode, power cycle your flight control board.**
+**To exit serial passthrough mode, power cycle your flight control board.**
+
+In order to reflash an Arduino based device such as a MWOSD via `serialpassthrough` if is necessary to connect the DTR line in addition to the RX and TX serial lines. The DTR is used as a reset line to invoke the bootloader. The DTR line may be connected to any GPIO pin on the flight control board. This pin must then be associated with a PINIO resource, the instance of which is then passed to the serialpassthrough command. The DTR line associated with any given UART may be set using the CLI command `resource` specifying it as a PINIO resource.
+
+For example, the following configuration for an OpenPilot Revolution shows the UART6 serial port to be configured with TX on pin C06, RX on pin C07 and a DTR connection using PINIO on pin C08.
+
+```
+resource SERIAL_TX 1 A09
+resource SERIAL_TX 3 B10
+resource SERIAL_TX 4 A00
+resource SERIAL_TX 6 C06
+resource SERIAL_RX 1 A10
+resource SERIAL_RX 3 B11
+resource SERIAL_RX 6 C07
+
+resource PINIO 1 C08
+```
+
+To assign the DTR line to another pin use the following command.
+
+```
+resource PINIO 1 c05
+```
+
+To disassociate DTR from a pin use the following command.
+
+```
+resource PINIO 1 none
+```
+
+Having configured a PINIO resource assocaited with a DTR line as per the above example, connection to an MWOSD attached to an Openpilot Revolution could be achieved using the following command.
+
+```serialpassthrough 5 0 rxtx 1```
+
+This will connect using UART 6, with the baud rate set over USB, full duplex, and with DTR driven on PINIO resource 1.
+
+A (desirable) side effect of configuring the DTR line to be associated with a PINIO resource, is that when the FC is reset, the attached Arduino device will also be reset.
+
+Note that if DTR is left configured on a port being used with a standard build of MWOSD firmware, the display will break-up when the flight controller is reset. This is because, by default, the MWOSD does not correctly handle resets from DTR. There are two solutions to this:
+
+1. Assign the DTR pin using the resource command above prior to reflashing MWOSD, and then dissasociate DTR from the pin.
+2. Rebuild MWOSD with MAX_SOFTRESET defined. The MWOSD will then be reset correctly every time the flight controller is reset.
+
+
+
+

lib/main/STM32_USB_Device_Library/Class/cdc/src/usbd_cdc_core.c

@@ -547,7 +547,7 @@ uint8_t usbd_cdc_Setup (void *pdev,
  else /* No Data request */
  {
  /* Transfer the command to the interface layer */
- APP_FOPS.pIf_Ctrl(req->bRequest, NULL, 0);
+ APP_FOPS.pIf_Ctrl(req->bRequest, (uint8_t *)&req->wValue, sizeof(req->wValue));
  }
 
  return USBD_OK;

src/main/drivers/serial.c

@@ -86,6 +86,24 @@ void serialSetMode(serialPort_t *instance, portMode_e mode)
  instance->vTable->setMode(instance, mode);
 }
 
+void serialSetCtrlLineStateCb(serialPort_t *serialPort, void (*cb)(void *context, uint16_t ctrlLineState), void *context)
+{
+ // If a callback routine for changes to control line state is supported by the underlying
+ // driver, then set the callback.
+ if (serialPort->vTable->setCtrlLineStateCb) {
+ serialPort->vTable->setCtrlLineStateCb(serialPort, cb, context);
+ }
+}
+
+void serialSetBaudRateCb(serialPort_t *serialPort, void (*cb)(serialPort_t *context, uint32_t baud), serialPort_t *context)
+{
+ // If a callback routine for changes to baud rate is supported by the underlying
+ // driver, then set the callback.
+ if (serialPort->vTable->setBaudRateCb) {
+ serialPort->vTable->setBaudRateCb(serialPort, cb, context);
+ }
+}
+
 void serialWriteBufShim(void *instance, const uint8_t *data, int count)
 {
  serialWriteBuf((serialPort_t *)instance, data, count);

src/main/drivers/serial.h

@@ -48,6 +48,10 @@ typedef enum {
  SERIAL_BIDIR_NOPULL = 1 << 5, // disable pulls in BIDIR RX mode
 } portOptions_e;
 
+// Define known line control states which may be passed up by underlying serial driver callback
+#define CTRL_LINE_STATE_DTR (1 << 0)
+#define CTRL_LINE_STATE_RTS (1 << 1)
+
 typedef void (*serialReceiveCallbackPtr)(uint16_t data, void *rxCallbackData); // used by serial drivers to return frames to app
 
 typedef struct serialPort_s {
@@ -105,6 +109,8 @@ struct serialPortVTable {
  bool (*isSerialTransmitBufferEmpty)(const serialPort_t *instance);
 
  void (*setMode)(serialPort_t *instance, portMode_e mode);
+ void (*setCtrlLineStateCb)(serialPort_t *instance, void (*cb)(void *instance, uint16_t ctrlLineState), void *context);
+ void (*setBaudRateCb)(serialPort_t *instance, void (*cb)(serialPort_t *context, uint32_t baud), serialPort_t *context);
 
  void (*writeBuf)(serialPort_t *instance, const void *data, int count);
  // Optional functions used to buffer large writes.
@@ -119,6 +125,8 @@ void serialWriteBuf(serialPort_t *instance, const uint8_t *data, int count);
 uint8_t serialRead(serialPort_t *instance);
 void serialSetBaudRate(serialPort_t *instance, uint32_t baudRate);
 void serialSetMode(serialPort_t *instance, portMode_e mode);
+void serialSetCtrlLineStateCb(serialPort_t *instance, void (*cb)(void *context, uint16_t ctrlLineState), void *context);
+void serialSetBaudRateCb(serialPort_t *instance, void (*cb)(serialPort_t *context, uint32_t baud), serialPort_t *context);
 bool isSerialTransmitBufferEmpty(const serialPort_t *instance);
 void serialPrint(serialPort_t *instance, const char *str);
 uint32_t serialGetBaudRate(serialPort_t *instance);

src/main/drivers/serial_escserial.c

@@ -842,6 +842,8 @@ const struct serialPortVTable escSerialVTable[] = {
  .serialSetBaudRate = escSerialSetBaudRate,
  .isSerialTransmitBufferEmpty = isEscSerialTransmitBufferEmpty,
  .setMode = escSerialSetMode,
+ .setCtrlLineStateCb = NULL,
+ .setBaudRateCb = NULL,
  .writeBuf = NULL,
  .beginWrite = NULL,
  .endWrite = NULL

src/main/drivers/serial_softserial.c

@@ -635,6 +635,8 @@ static const struct serialPortVTable softSerialVTable = {
  .serialSetBaudRate = softSerialSetBaudRate,
  .isSerialTransmitBufferEmpty = isSoftSerialTransmitBufferEmpty,
  .setMode = softSerialSetMode,
+ .setCtrlLineStateCb = NULL,
+ .setBaudRateCb = NULL,
  .writeBuf = NULL,
  .beginWrite = NULL,
  .endWrite = NULL

src/main/drivers/serial_tcp.c

@@ -263,6 +263,8 @@ static const struct serialPortVTable tcpVTable = {
  .serialSetBaudRate = NULL,
  .isSerialTransmitBufferEmpty = isTcpTransmitBufferEmpty,
  .setMode = NULL,
+ .setCtrlLineStateCb = NULL,
+ .setBaudRateCb = NULL,
  .writeBuf = NULL,
  .beginWrite = NULL,
  .endWrite = NULL,

src/main/drivers/serial_uart.c

@@ -274,6 +274,8 @@ const struct serialPortVTable uartVTable[] = {
  .serialSetBaudRate = uartSetBaudRate,
  .isSerialTransmitBufferEmpty = isUartTransmitBufferEmpty,
  .setMode = uartSetMode,
+ .setCtrlLineStateCb = NULL,
+ .setBaudRateCb = NULL,
  .writeBuf = NULL,
  .beginWrite = NULL,
  .endWrite = NULL,

src/main/drivers/serial_uart_hal.c

@@ -361,6 +361,8 @@ const struct serialPortVTable uartVTable[] = {
  .serialSetBaudRate = uartSetBaudRate,
  .isSerialTransmitBufferEmpty = isUartTransmitBufferEmpty,
  .setMode = uartSetMode,
+ .setCtrlLineStateCb = NULL,
+ .setBaudRateCb = NULL,
  .writeBuf = NULL,
  .beginWrite = NULL,
  .endWrite = NULL,

src/main/drivers/serial_usb_vcp.c

@@ -67,6 +67,22 @@ static void usbVcpSetMode(serialPort_t *instance, portMode_e mode)
  // TODO implement
 }
 
+static void usbVcpSetCtrlLineStateCb(serialPort_t *instance, void (*cb)(void *context, uint16_t ctrlLineState), void *context)
+{
+ UNUSED(instance);
+
+ // Register upper driver control line state callback routine with USB driver
+ CDC_SetCtrlLineStateCb((void (*)(void *context, uint16_t ctrlLineState))cb, context);
+}
+
+static void usbVcpSetBaudRateCb(serialPort_t *instance, void (*cb)(serialPort_t *context, uint32_t baud), serialPort_t *context)
+{
+ UNUSED(instance);
+
+ // Register upper driver baud rate callback routine with USB driver
+ CDC_SetBaudRateCb((void (*)(void *context, uint32_t baud))cb, (void *)context);
+}
+
 static bool isUsbVcpTransmitBufferEmpty(const serialPort_t *instance)
 {
  UNUSED(instance);
@@ -178,6 +194,8 @@ static const struct serialPortVTable usbVTable[] = {
  .serialSetBaudRate = usbVcpSetBaudRate,
  .isSerialTransmitBufferEmpty = isUsbVcpTransmitBufferEmpty,
  .setMode = usbVcpSetMode,
+ .setCtrlLineStateCb = usbVcpSetCtrlLineStateCb,
+ .setBaudRateCb = usbVcpSetBaudRateCb,
  .writeBuf = usbVcpWriteBuf,
  .beginWrite = usbVcpBeginWrite,
  .endWrite = usbVcpEndWrite

src/main/interface/cli.c

@@ -917,6 +917,15 @@ static void cliSerial(char *cmdline)
 }
 
 #ifndef SKIP_SERIAL_PASSTHROUGH
+#ifdef USE_PINIO
+static void cbCtrlLine(void *context, uint16_t ctrl)
+{
+ int pinioDtr = (int)(long)context;
+
+ pinioSet(pinioDtr, !(ctrl & CTRL_LINE_STATE_DTR));
+}
+#endif /* USE_PINIO */
+
 static void cliSerialPassthrough(char *cmdline)
 {
  if (isEmpty(cmdline)) {
@@ -926,6 +935,10 @@ static void cliSerialPassthrough(char *cmdline)
 
  int id = -1;
  uint32_t baud = 0;
+ bool enableBaudCb = false;
+#ifdef USE_PINIO
+ int pinioDtr = 0;
+#endif /* USE_PINIO */
  unsigned mode = 0;
  char *saveptr;
  char* tok = strtok_r(cmdline, " ", &saveptr);
@@ -945,21 +958,32 @@ static void cliSerialPassthrough(char *cmdline)
  if (strstr(tok, "tx") || strstr(tok, "TX"))
  mode |= MODE_TX;
  break;
+#ifdef USE_PINIO
+ case 3:
+ pinioDtr = atoi(tok);
+ break;
+#endif /* USE_PINIO */
  }
  index++;
  tok = strtok_r(NULL, " ", &saveptr);
  }
 
+ if (baud == 0) {
+ enableBaudCb = true;
+ }
+
  cliPrintf("Port %d ", id);
  serialPort_t *passThroughPort;
  serialPortUsage_t *passThroughPortUsage = findSerialPortUsageByIdentifier(id);
  if (!passThroughPortUsage || passThroughPortUsage->serialPort == NULL) {
- if (!baud) {
- cliPrintLine("closed, specify baud.");
- return;
+ if (enableBaudCb) {
+ // Set default baud
+ baud = 57600;
  }
- if (!mode)
+
+ if (!mode) {
  mode = MODE_RXTX;
+ }
 
  passThroughPort = openSerialPort(id, FUNCTION_NONE, NULL, NULL,
  baud, mode,
@@ -968,26 +992,54 @@ static void cliSerialPassthrough(char *cmdline)
  cliPrintLine("could not be opened.");
  return;
  }
- cliPrintf("opened, baud = %d.\r\n", baud);
+
+ if (enableBaudCb) {
+ cliPrintf("opened, default baud = %d.\r\n", baud);
+ } else {
+ cliPrintf("opened, baud = %d.\r\n", baud);
+ }
  } else {
  passThroughPort = passThroughPortUsage->serialPort;
  // If the user supplied a mode, override the port's mode, otherwise
  // leave the mode unchanged. serialPassthrough() handles one-way ports.
- cliPrintLine("already open.");
+ // Set the baud rate if specified
+ if (baud) {
+ cliPrintf("already open, setting baud = %d.\n\r", baud);
+ serialSetBaudRate(passThroughPort, baud);
+ } else {
+ cliPrintf("already open, baud = %d.\n\r", passThroughPort->baudRate);
+ }
+
  if (mode && passThroughPort->mode != mode) {
- cliPrintf("mode changed from %d to %d.\r\n",
+ cliPrintf("Mode changed from %d to %d.\r\n",
  passThroughPort->mode, mode);
  serialSetMode(passThroughPort, mode);
  }
+
  // If this port has a rx callback associated we need to remove it now.
  // Otherwise no data will be pushed in the serial port buffer!
  if (passThroughPort->rxCallback) {
  passThroughPort->rxCallback = 0;
  }
  }
 
+ // If no baud rate is specified allow to be set via USB
+ if (enableBaudCb) {
+ cliPrintLine("Baud rate change over USB enabled.");
+ // Register the right side baud rate setting routine with the left side which allows setting of the UART
+ // baud rate over USB without setting it using the serialpassthrough command
+ serialSetBaudRateCb(cliPort, serialSetBaudRate, passThroughPort);
+ }
+
  cliPrintLine("Forwarding, power cycle to exit.");
 
+#ifdef USE_PINIO
+ // Register control line state callback
+ if (pinioDtr) {
+ serialSetCtrlLineStateCb(cliPort, cbCtrlLine, (void *)(intptr_t)(pinioDtr - 1));
+ }
+#endif /* USE_PINIO */
+
  serialPassthrough(cliPort, passThroughPort, NULL, NULL);
 }
 #endif
@@ -3753,7 +3805,7 @@ const clicmd_t cmdTable[] = {
 #endif
  CLI_COMMAND_DEF("serial", "configure serial ports", NULL, cliSerial),
 #ifndef SKIP_SERIAL_PASSTHROUGH
- CLI_COMMAND_DEF("serialpassthrough", "passthrough serial data to port", "<id> [baud] [mode] : passthrough to serial", cliSerialPassthrough),
+ CLI_COMMAND_DEF("serialpassthrough", "passthrough serial data to port", "<id> [baud] [mode] [DTR PINIO]: passthrough to serial", cliSerialPassthrough),
 #endif
 #ifdef USE_SERVOS
  CLI_COMMAND_DEF("servo", "configure servos", NULL, cliServo),

src/main/io/serial.c

@@ -58,6 +58,8 @@
 #include "telemetry/telemetry.h"
 #endif
 
+#include "pg/pinio.h"
+
 static serialPortUsage_t serialPortUsageList[SERIAL_PORT_COUNT];
 
 const serialPortIdentifier_e serialPortIdentifiers[SERIAL_PORT_COUNT] = {
@@ -429,7 +431,6 @@ void closeSerialPort(serialPort_t *serialPort)
  }
 
  // TODO wait until data has been transmitted.
-
  serialPort->rxCallback = NULL;
 
  serialPortUsage->function = FUNCTION_NONE;
@@ -540,13 +541,17 @@ void serialPassthrough(serialPort_t *left, serialPort_t *right, serialConsumer *
  if (serialRxBytesWaiting(left)) {
  LED0_ON;
  uint8_t c = serialRead(left);
+ // Make sure there is space in the tx buffer
+ while (!serialTxBytesFree(right));
  serialWrite(right, c);
  leftC(c);
  LED0_OFF;
  }
  if (serialRxBytesWaiting(right)) {
  LED0_ON;
  uint8_t c = serialRead(right);
+ // Make sure there is space in the tx buffer
+ while (!serialTxBytesFree(left));
  serialWrite(left, c);
  rightC(c);
  LED0_OFF;

src/main/target/REVO/target.h

@@ -217,6 +217,7 @@
 #define USE_UART6
 #define UART6_RX_PIN PC7
 #define UART6_TX_PIN PC6
+#define PINIO1_PIN PC8 // DTR pin
 
 #define USE_SOFTSERIAL1
 #define USE_SOFTSERIAL2

src/main/target/REVONANO/target.h

@@ -56,6 +56,7 @@
 #define USE_UART1 // Flexi Port
 #define UART1_RX_PIN PB7
 #define UART1_TX_PIN PB6
+#define PINIO1_PIN PB10 // DTR pin
 
 #define USE_UART2 // Main Port
 #define UART2_RX_PIN PA3