Modifying the atmega16u2 firmware on the Arduino Due

due-leo_g

Flashing new firmware into the Due’s atmega16u2, using a Leonardo as ISP.

Compiling the firmware

The arduino ide comes with the sources for the 16u2’s firmware. It links against  LUFA  (Lightweight USB framework for AVR’s) which you need to download first. Look for version 100807, not the latest one: the 16u2 firmware from the current ide (1.5.2) does not build as is with the latest version of LUFA. Unzip the zip file in any location on your PC.

Locate the 16u2 firmware directory in the arduino install dir: arduino-1.5.2/hardware/arduino/sam/firmwares/atmega16u2/arduino-usbserial. Copy it into LUFA’s Project sub directory, so you end up with a directory LUFA100807/Projects/arduino-usbserial. In that directory, simply type ‘make’.

In a next post I want to present a modified firmware that allows you to disable autoreset on the Due’s programming USB port…

Flashing the new firmware

There is a tutorial on how to flash new firmware in the atmega16u2 on the Due here. However, the wiring used over there uses pins 10, 11, 12 and 13 on the Arduino that serves as programmer. That does not work on the Leonardo. With the info from my first post, it is straightforward to do this task with a Leonardo too. So following diagram does not bring world shocking new information, but I posted it anyway as I found it practical. Pay attention to the orientation of the Due: its isp header is “upside down” as compared to the SPI header.

due-leo-schema_g

This wiring works on classic Arduino’s like the Uno or Duemilanove too. I find this one easier to remember and to wire.

The procedure in the official tutorial can be used as is with the Leonardo. Except that there is no need to disable autoreset. The procedure is stable, there is not a lot that can go wrong. And if it fails you can try again. Nevertheless I recommend to first try to download the current firmware from the device, just to verify you have a working isp (e.g. for linux, from the ide’s tools dir):
  ./avrdude -C avrdude.conf -c arduino -P /dev/ttyACM0 -b 19200 -p m16u2 -vvv -U flash:r:old.hex:i

I also recommend to install the three leds. If the sketch hangs somewhere (which happens rarely though), you see immedeately what happened.

I did not connect the 5V pins. Both boards are powered by usb. You can compile a new firmware and after upload, the Leo drives nRESET high and the Due is immedeately ready for action again.

Advertisements

Using the Leonardo as USB to serial converter.

Arduino Leonardo used as usb to serial converter.

The green board in the picture above carries an atmega 328p with a Duemilanove bootloader. The Leonardo works as an USB to serial converter and is used to upload sketches to the atmega328p, and to communicate serially with the sketches.

For serial communications and power, the green board has the common 6 pin header you also find on e.g. the Arduino Pro or the Sanguino:

Target processor      Pin                           Serial to usb converter
(Atmega 328p)         Header                        (Leonardo)
GND   ---------------|1|--------------------------- GND
                     |2|                           
5V    <--------------|3|--------------------------- 5V
RX    <--- 1K -------|4|--------------------------- TX
TX    ---- 1K -------|5|--------------------------> RX
RESET <----||--------|6|--------------------------- DTR_PIN (here pin 13 4)
  |       100nF    
 10K
  |
 5V

The DTR pin is used by the Arduino IDE to automatically trigger a reset of the target processor, when it uploads a sketch. The header is meant to be used with an usb to serial converter like e.g. Sparkfun’s  “FTDI simple break out board”. This post is about a sketch and a few hacks in the arduino core that allows you to use the Leonardo  for this purpose.

UPDATE (September 2015): Thanks to Matthijs Kooiman’s work (see Expose CDC settings to sketch #3343), it is now possible to have this functionality without having to hack the arduino core.
Since long before the Leonardo was released (!), Paul Stoffregen has a more optimized sketch. He posted an updated version here: https://github.com/arduino/Arduino/pull/3343#issuecomment-115045979. You need Arduino 1.6.6.

I leave the rest of this post around  as people have used it and come back here. But I believe it is now obsolete, even for work on older cores. You have to modify those cores anyway and therefore I think it is better to add the Serial_.dtr() and Serial_.baud() methods to the old core and rewrite the sketch below without the callbacks. The callbacks below run in interrupt mode. It is difficult to guarantee such a callback is synchronized correctly with normal code. E.g. a flaw in the sketch below is that the loop() function may read a corrupted value of variable ‘baud’. In practice this will not cause a problem as in the next pass through loop(), the baud rate will be set correctly. But this illustrates it is simpler to avoid callbacks under interrupt…
____________________________________

Here is the sketch:
(again: it makes more sense to use the sketch from the link above).

/*
  leo_usb2serial
  Allows to use an Arduino Leonardo as an usb to serial converter.
 */
static long baud = 57600;
static long newBaud = baud;

// this pin will output the DTR signal (as set by the pc)
#define DTR_PIN 13

#define LINESTATE_DTR  1

void lineCodingEvent(long baud, byte databits, byte parity, byte charFormat)
{
  newBaud = baud;
}

void lineStateEvent(unsigned char linestate)
{
  if(linestate & LINESTATE_DTR)
    digitalWrite(DTR_PIN, HIGH);
  else
    digitalWrite(DTR_PIN, LOW);
}

void setup() {
  pinMode(DTR_PIN, OUTPUT);
  digitalWrite(DTR_PIN, LOW);
  Serial.begin(baud);
  Serial1.begin(baud);
}

void loop() {

  // Set the new baud rate
  if(newBaud != baud) {
    baud = newBaud;
    Serial1.end();
    Serial1.begin(baud);
  }

  // copy from virtual serial line to uart and vice versa
  if (Serial.available()) {
    char c = (char)Serial.read();
    Serial1.write(c);
  }
  if (Serial1.available()) {
    char c = (char)Serial1.read();
    Serial.write(c);
  }
}

The sketch’s main job is to forward everything that is received over USB (Serial) onto the real uart (Serial1), and vice versa. That is what happens at the end of loop().

Another task the sketch has to accomplish is to update the uart’s baudrate, whenever the pc changes the baud rate of the virtual com port. When this happens, the arduino core calls lineCodingEvent(baud,...). This routine runs under interrupt so we must take care not to spend too much time in it. Therefore the new baud rate is recorded in newBaud and the actual work is done from loop():

 if(newBaud != baud) {
    baud = newBaud;
    Serial1.end();
    Serial1.begin(baud);
  }

The last thing is to make the DTR signal available on one of the digital pins. Whenever the pc sets or clears DTR of the virtual com port, the arduino core calls lineStateEvent(). This routine also runs under interrupt, but the only thing to do is to adjust the level of the DTR_PIN. Pin 13 is used as DTR_PIN, this way one can observe the led to see what the Arduino IDE does with the DTR signal.

Now we get to the hacking part. Neither lineCodingEvent() nor lineStateEvent() are part of the arduino core. I plan to submit a change request for this, it looks a useful feature to me. However, it is not intrusive to add this feature manually to the core. Besides the changes discussed in my previous post are also needed for this to work, otherwise serial buffer overruns will happen.

Locate the CDC.cpp file and add the lines printed in bold:

void WEAK lineCodingEvent(long baud, byte databits, byte parity, byte charFormat)
{
}

void WEAK lineStateEvent(byte linestate)
{
}

bool WEAK CDC_Setup(Setup& setup)
{
        u8 r = setup.bRequest;
        u8 requestType = setup.bmRequestType;

        if (REQUEST_DEVICETOHOST_CLASS_INTERFACE == requestType)
        {
                if (CDC_GET_LINE_CODING == r)
                {
                        USB_SendControl(0,(void*)&_usbLineInfo,7);
                        return true;
                }
        }

        if (REQUEST_HOSTTODEVICE_CLASS_INTERFACE == requestType)
        {
                if (CDC_SET_LINE_CODING == r)
                {
                        USB_RecvControl((void*)&_usbLineInfo,7);
                        lineCodingEvent(_usbLineInfo.dwDTERate,
                                        _usbLineInfo.bDataBits,
                                        _usbLineInfo.bParityType,
                                        _usbLineInfo.bCharFormat);
                        return true;
                }

                if (CDC_SET_CONTROL_LINE_STATE == r)
                {
                        _usbLineInfo.lineState = setup.wValueL;

                        lineStateEvent(_usbLineInfo.lineState);

                        // auto-reset into the bootloader is triggered when the port, already 
                        // open at 1200 bps, is closed.  this is the signal to start the watchdog
                        // with a relatively long period so it can finish housekeeping tasks
                        // like servicing endpoints before the sketch ends
                        if (1200 == _usbLineInfo.dwDTERate) {
                                // We check DTR state to determine if host port is open (bit 0 of lineState).
                                if ((_usbLineInfo.lineState & 0x01) == 0) {
                                        *(uint16_t *)0x0800 = 0x7777;
                                        wdt_enable(WDTO_120MS);
                                } else {
                                        // Most OSs do some intermediate steps when configuring ports and DTR can
                                        // twiggle more than once before stabilizing.
                                        // To avoid spurious resets we set the watchdog to 250ms and eventually
                                        ...

First, weak symbols are add for lineCodingEvent() and lineStateEvent(). This makes sure that if a sketch does not provide one of the functions, the linker will use these dummy ones.

Then, it suffices to call the functions at the right place. CDC_Setup() is called whenever the Leonardo receives a “control message” , related to the CDC (Communication Device Class) protocol. Line coding and line state are set via such control messages. In the above snippet, the event functions are called after the messages are parsed.

I tried out the sketch by downloading the asciiTable sample using the IDE. The target atmega328p gets autoreset and the baudrate updated to 57600, which is what the Duemilanove bootloader uses. When the download completed, I opened the serial monitor and set it to 9600 baud. Upon reset of the atmega328p, I received the expected output from the asciiTable sketch.

ArduinoISP on the Leonardo

A Leonardo programming an attiny2313

A while ago I noticed this thread about burning an Atmega 328 bootloader with an Arduino Leonardo. It turned out that the ArduinoISP sketch that comes with the arduino ide (version 1.0.1) does not work out of the box on the recent Leonardo model. I decided to buy one and help investigating the problems reported in that thread. Following issues were posted:

In mean time fixes for these issues are committed in the development tree so I expect them to become available in an upcoming release of the ide. Untill then, here is a wrap up on how to use the leonardo as an isp programmer with Arduino 1.0.1. Edit: the fix proposed below in step 3 made it into Arduino 1.0.2, so it is no longer needed. The rest is still applicable.

1. Like in the picture above, connect the target’s MISO/MOSI and SCK signals to the ICSP header of the Leonardo. Don’t connect them to digital pins 11, 12 and 13: on the leo there is no SPI on these pins.

Edit: the picture above is just for setting the scene. Ja450n (thank you) sent me a nice Fritzing drawing, featuring an attiny85,  that is more illustrative:
leo_attiny85

2. In ArduinoISP.ino, change this line:
#define RESET SS
into:
#define RESET 10

Edit November 2015: Since Arduino 1.6.6 this change is already in the IDE.

This states we want to use pin 10, to reset the target mcu .(the brown wire in the picture).
On the leo, SS is not available on arduino pin 10. Actually it is not available on any arduino pin, the atmega pin that exposes SS is used to drive the RX led. Fortunately it is perfectly ok to use pin 10 (or any other digital pin) to drive the target’s reset. (The only requirement for the SS pin is that it is configured as an output (which makes the leo an SPI master). This requirement is fulfilled since it drives the rx led.

3. Locate the CDC.cpp and USBCore.cpp files in your arduino installation directory and apply following modifications (you may want to make a copy first):
In CDC.cpp, the accept routine should look like this:

void Serial_::accept(void)
{
    ring_buffer *buffer = &cdc_rx_buffer;

    int i = (unsigned int)(buffer->head+1) % SERIAL_BUFFER_SIZE;

    // if we should be storing the received character into the location
    // just before the tail (meaning that the head would advance to the
    // current location of the tail), we're about to overflow the buffer
    // and so we don't write the character or advance the head.

    // while we have room to store a byte
    while (i != buffer->tail) {
        int c = USB_Recv(CDC_RX);
        if (c == -1)
            break; // no more data, we're done
        buffer->buffer[buffer->head] = c;
        buffer->head = i;

        i = (unsigned int)(buffer->head+1) % SERIAL_BUFFER_SIZE;
    }
}

In USBCore.cpp, search following snippet:

#ifdef CDC_ENABLED
    USB_Flush(CDC_TX); // Send a tx frame if found
    while (USB_Available(CDC_RX)) // Handle received bytes (if any)
        Serial.accept();
#endif

and change it into:

#ifdef CDC_ENABLED
    USB_Flush(CDC_TX); // Send a tx frame if found
    if (USB_Available(CDC_RX)) // Handle received bytes (if any)
        Serial.accept();
#endif

4. Compile and upload ArduinoISP to the leo.

5. In the ide, open the serial monitor and see to it that the baud rate is not equal to 1200bps. Close the serial monitor again. This step removes the magic baud rate from the serial port which would cause unintentional auto resets.

6. On linux (and probably mac, though I don’t have a mac to try it) you can just go to the “Tools > Programmer” menu and select “Arduino as ISP” as programmer, Now the leo is ready to act as an isp. From here, follow the regular procedure to burn a bootloader in the target. Or you can use avrdude from the command line to burn any hex file in the target…

7. On windows an extra step is required. When selecting “Arduino as ISP” as programmer, the IDE will invoke avrdude and instruct it to use the stk500v1 protocol. Currently (arduino 1.0.2 and 1.0.3) this does not work on windows. It will result in following error message:

avrdude.exe: Send: 0 [30]   [20]
avrdude.exe: Send: 0 [30]   [20]
avrdude.exe: Send: 0 [30]   [20]
avrdude.exe: Recv:
avrdude.exe: stk500_getsync(): not in sync: resp=0x00

Note that avrdude sends out stuff, but does not receive anything. The leo’s rx led will flash but the tx led stays dark.

As a  temporary work around it was suggested (see here) to define a programmer that instructs avrdude to use the “arduino” protocol instead of “stk500v1”. To do so, create a sub directory named “hardware/leofix” in your sketchbook directory. In that directory install a file “programmers.txt” with following content:

arduinoispleo.name=Arduino as ISP (Leonardo)
arduinoispleo.communication=serial
arduinoispleo.protocol=arduino
arduinoispleo.speed=19200

After restarting the IDE, you will have an entry “Arduino as ISP (Leonardo)” in the “Tools > Programmers” menu. This will do the job on windows too.

The reason stk500v1 does not work is that upon opening the virtual com port, windows/avrdude do not request to assert the “virtual” DTR and RTS signals. The leonardo refuses to send out data under these conditions so it never writes data back to avrdude. When using the arduino protocol, before starting programming, avrdude will briefly deassert DTR and RTS and assert them again. This would trigger autoreset if you were uploading a sketch to an arduino bootloader. But in this work around, we use the side effect that this leaves DTR/RTS asserted during programming which makes the leonardo talk..

I filed an issue for this too:

8. Another reason why ArduinoISP may not work, may come from the fact that the Leonardo is a composite device (it combines a virtual comm port and a HID (keyboard/mouse) in one USB device). Obviously, if you are on an old operating system that does not support composite devices, ArduinoISP will not work, nor will any sketch that uses USB. (Confusingly, uploading sketches is possible because the boot loader is not a composite device.)

Windows has support from XP SP3 on and from Vista SP1 on, Mac has support since Lion, see here for more background.

Since ArduinoISP does not need the HID stuff, a work around is to build it as a “serial port only” sketch. This is described here.