ArduinoISP on the Due

There may not be a lot of practical value in running ArduinoISP on an Aruino Due. If you have a Due, it is very likely you have at least one classic Arduino at hand for which running ArduinoISP is well known and documented. On the other hand, ArduinoISP is a standard Arduino sample so I find that it should work on any Arduino. Well, it was good fun to make it work…

Due programming bootloader into a Leonardo

Due programming bootloader into a Leonardo

I found an opportunity to start with this when I wanted to upgrade the firmware of my Leonardo

Since Arduino 1.0.3, there is an improved boot loader for Leonardo that is a real must have: upon power-on reset, it jumps immediately to your sketch instead of awaiting programmer commands for some seconds.

Just for the sake of it I decided use my Due for this task.

Due programming an attiny85

Due programming an attiny85

Later on, I tried to program an attiny85. This required some extra measures. The Due’s SPI runs too fast for these targets so I had to use software SPI to slow the Due down.

Wiring, 5V, 3V3…

First thing to take care of was to make sure not to fry the Due which is not 5V tolerant. Connecting it to the ISP port of the Leonardo powered with 5V would probably damage the Due: the Leonardo would drive the MISO signal to 5V. I took a very simple approach: I powered the Leonardo with 3.3V. See red jumper wire between the Due’s 3V3 (don’t accidentally use the adjacent 5V pin) pin and the Leo’s 5V pin.

As usual, my photo’s are lousy, but these are the connections:

Due     | Target (Leonardo, attiny85...)
 MISO   |  MISO
 SCK    |  SCK
 MOSI   |  MOSI
 GND    |  GND
 3.3V   |  Vcc (labeled 5V, on the Leonardo)
 Dig 10 |  RESET

It should be warned not to connect the 5V (!) power pin from the Due’s SPI header to the corresponding pin on the Leonardo (using a flat cable would do this). Like most people on the forum I expected this pin to carry 3V3, and first I was tempted to use it to power the Leonardo. Well it has 5V. OK, trap avoided.

Also when this wiring is in place, don’t connect the Leonardo’s USB cable !

In general, don’t power the target at 5V when doing this.

Get a “Due – ready” version of ArduinoISP

The version of ArduinoISP that comes with the ide is a bit outdated. It does not even compile for the Due because the SPI functionality is implemented using AVR specific SPI registers.

I keep a version here that works on the Due without need for further tweaking, nor patches in the core.

I plan to write more about its details in a later post. It has some options you can configure and it has a few new fixes.

Also for other Arduino’s, this version works out of the box, with a reasonable default configuration. Therefore I think it is a candidate for inclusion in the IDE.  We’ll see…

For now I just wish to thank Sylvan Butler, whose SPI bitbang implementation I started from.

Try it out.

Compile and upload the sketch with arduino 1.5.2 to your Due. Once that is done it is ready to do ISP. Use the native USB port (see photo).

From this point on, when it comes to programming a target, it makes more sense to use arduino 1.0.x. This is OK, the IDE does not care your ArduinoISP programmer is actually a Due.

Windows users have one more thing to take care of: the ide must be instructed to use “arduino” as programmer instead of “stk500v1″. It is explained here (go to point 7), but looking back on it, I find it more easy to just go into the arduino-1.0.x\hardware\arduino\programmers.txt and change the entry:
   arduinoisp.protocol=stk500v1
into:
arduinoisp.protocol=arduino

That is it. I tested burning the bootloader into my Leonardo using arduino 1.0.3 on linux (Ubuntu 12.04.2 LTS) and on Windows 7 SP1 64bit. I also burned a bootloader into a 328p clocked with16MHz.

It was also possible to burn some led blinking firmwares into an attiny85 clocked@1MHz, internal oscillator.

Disabling auto reset on the Arduino Due

jumper

Autoreset disable jumper
(modified 16u2 firmware)

This post is about a simple modification to the firmware of the atmega16u2 on the Arduino Due board, that makes it possible to disable auto reset on the Due’s programming USB port.

Edit: as Topaz replied, there is a much easier way to disable auto reset: put a 1K resistor between reset and 3V3. Nevertheless the firmware mod described here has the convenience that it disables auto reset while auto upload still works. –  And it is good fun too!

I found this feature was best introduced via a small tour of observations about auto reset on various Arduino types.

Auto reset on classic Arduinos.

On classic Arduinos, the nDTR output pin of the USB to serial converter is connected to the AVR’s nRESET pin. If you start avrdude to upload a sketch, the serial line is opened and the operating system normally brings nDTR low. This is because with a classic modem, nDTR (Data Terminal Ready) indicates to the modem that the “terminal” (the PC) is ready to  for communication. So connecting nDTR to nRESET automatically resets the AVR.

There is a serial capacitor between nDTR and nRESET that starts charging up (through nRESET’s 10K pullup resistor) from the moment nDTR goes low, thus providing for a short reset pulse.

Further there is a diode from nRESET to 5V. This is because when nDTR goes back high, there is momentarily 10V on nRESET. Absence of such a diode was reported to cause unintentional high voltage programming on the AVR.

Sometimes the RTS signal is used instead of DTR.

Most Arduino’s have a trace that can be cut to disable auto reset. On my Duemilanove I soldered a jumper across the trace so I can easily close the trace again if I want auto reset.

Another method mentioned often in the fora is to place a 20uF cap between nRESET and ground that prevents nRESET from being pulled low. See here.

Why disabling auto reset?

The side effect of auto reset is that if you you open the serial port just to see the output of your program, the processor is reset as well. In many (if not most) situations, this is not what you want. However, it looks like over time, Arduino users started considering this behavior as normal and even desired. This is because if the processor is reset and thus the sketch is restarted, the messages sent to the serial line in the beginning of the sketch will nicely show up in your terminal program.

A related problem with auto reset is that when software on your PC opens the serial port and sends data to it, the Arduino resets, its bootloader starts running and it will consume the first bytes sent by the PC…

The Leonardo’s USB port.

The Leonardo has a different behaviour. It does not auto reset if you open the serial port. When the IDE wants to upload a sketch, it has to indicate this intent by “touching” the port at the magic baud rate of 1200bps. Only then auto reset will happen. I find this behaviour more useful. But you lose the feature of catching early println’s in your terminal program. No big deal to me, but apparently it was important enough to for the developers come up with the

while (!Serial)
    ;

idiom. The idea is this: since the sketch is not restarted when the port is opened, we let the sketch wait until the port is opened.

If you have a look at how the boolean operator for class Serial is implemented you’ll see that it returns true when DTR or RTS are asserted i.e. when the port is “open”.

The Due’s native USB port.

The Due’s native USB port works much like the Leonardo’s. So if you want to avoid auto reset on the Due, the simplest way is to use this port. Use SerialUSB instead of Serial in your sketch.

The Due’s programming USB port.

The Due’s programming USB port is a real uart on the sam, connected to an uart on the atmegu16u2 which serves as serial to USB converter. To be able to flash a new sketch into the sam,  the 16u2 must perform a special sequence: it must first pulse high the sam’s erase line, then pulse low nRESET. Of coarse this is not done whenever the serial port is opened: this would erase the flash upon every opening of the serial port! So this erase+reset sequence is only done after the ide has indicated its intent to upload a sketch by opening the port at 1200 bps.

But what happens if you open the serial port at another baudrate than 1200 bps? The erase line is not touched, but what about the reset line?  Well, in this case the sam gets reset too! Clearly the designers opted to be consistent with the behaviour of the Uno, not that of the Leonardo.

Seen the similarity with the Uno, I tried disabling auto reset by installing a 20uF capacitor between the nRESET of the Due and ground. That does not work. Reason is that the Due has no 10nF capacitor between the 16u2’s output and the sam’s nRESET. Therefore, nRESET is pulled low much longer: 200msec. This is enough to drain the capacitor and reset happens.

By the way, there is no buffer between the 16u2’s output pin and the sam’s nRESET. This looks odd because it could damage the Due. Here, it does not hurt because the 16u2’s firmware never drives the pin high, it either pulls it low to reset the sam or configures it as input (high impedance) otherwise. In the latter case, the sam internally pulls up nRESET (to 3V3).

I find the use of a big capacitor to disable auto reset not a proper solution anyway. A jumper would be better. But there is no trace you can cut to disable autoreset like on the Uno.

So I decided to solder jumpers to the four free gpio’s of the 16u2 that are broken out. I modified the firmware such that it only carries out auto reset when a jumper bridge is in place between PB5 and PB7.due_16u2_gpio

See my previous post for where to find the code for the 16u2 firmware.

In Arduino-usbserial.c:

#define JUMPER_SENSE 5
#define JUMPER_GND 7
#define JUMPER_DEBUG 4

JUMPER_GND (PB7) is driven low and JUMPER_SENSE (PB5) senses whether it is jumpered (pulled low) to the first one. This is set up in SetupHardware():

...
/* Jumper pins */
PORTB |=  (1<<JUMPER_SENSE); // pull-up
DDRB  &= ~(1<<JUMPER_SENSE); // input
PORTB &= ~(1<<JUMPER_GND);   // low
DDRB  |=  (1<<JUMPER_GND);   // output

When the PC opens the port, this causes EVENT_CDC_Device_ControLineStateChanged() to get called. Lines in bold were add:

void EVENT_CDC_Device_ControLineStateChanged(USB_ClassInfo_CDC_Device_t* const CDCInterfaceInfo)
{
    ...
    if (Selected1200BPS) {
        /* Start Erase / Reset procedure when receiving the magic "1200" baudrate */
        ResetTimer = 120;
    } else if (!PreviousDTRState && CurrentDTRState) {
        /* Reset on rising edge of DTR
           but only if jumper is NOT in place */
        if (PINB & (1 << JUMPER_SENSE))
            ResetTimer = 30;
    }
}

So, if the line state changes (port opened or closed) at 1200 bps, the ResetTimer is set to 120. The main loop of the firmware will start counting down ResetTimer from 120 to zero, executing the erase+reset sequence in this process.

When opened at another baud rate, the ResetTimer will set to 30. As a consequence the main loop will only execute the last part of the sequence, which is the reset part. The firmware modification makes sure the reset part is only armed if the jumper is NOT in place.

The nice thing is that even with auto reset disabled, uploading sketches still works automatically! This is because when uploading, the magic 1200 bps baud rate is used which still arms the erase+reset sequence!

Modifying the atmega16u2 firmware on the Arduino Due

due-leo_g

Flashing new firmware into the Due’s atmega16u2, using a Leonardo as ISP.

Compiling the firmware

The arduino ide comes with the sources for the 16u2’s firmware. It links against  LUFA  (Lightweight USB framework for AVR’s) which you need to download first. Look for version 100807, not the latest one: the 16u2 firmware from the current ide (1.5.2) does not build as is with the latest version of LUFA. Unzip the zip file in any location on your PC.

Locate the 16u2 firmware directory in the arduino install dir: arduino-1.5.2/hardware/arduino/sam/firmwares/atmega16u2/arduino-usbserial. Copy it into LUFA’s Project sub directory, so you end up with a directory LUFA100807/Projects/arduino-usbserial. In that directory, simply type ‘make’.

In a next post I want to present a modified firmware that allows you to disable autoreset on the Due’s programming USB port…

Flashing the new firmware

There is a tutorial on how to flash new firmware in the atmega16u2 on the Due here. However, the wiring used over there uses pins 10, 11, 12 and 13 on the Arduino that serves as programmer. That does not work on the Leonardo. With the info from my first post, it is straightforward to do this task with a Leonardo too. So following diagram does not bring world shocking new information, but I posted it anyway as I found it practical. Pay attention to the orientation of the Due: its isp header is “upside down” as compared to the SPI header.

due-leo-schema_g

This wiring works on classic Arduino’s like the Uno or Duemilanove too. I find this one easier to remember and to wire.

The procedure in the official tutorial can be used as is with the Leonardo. Except that there is no need to disable autoreset. The procedure is stable, there is not a lot that can go wrong. And if it fails you can try again. Nevertheless I recommend to first try to download the current firmware from the device, just to verify you have a working isp (e.g. for linux, from the ide’s tools dir):
  ./avrdude -C avrdude.conf -c arduino -P /dev/ttyACM0 -b 19200 -p m16u2 -vvv -U flash:r:old.hex:i

I also recommend to install the three leds. If the sketch hangs somewhere (which happens rarely though), you see immedeately what happened.

I did not connect the 5V pins. Both boards are powered by usb. You can compile a new firmware and after upload, the Leo drives nRESET high and the Due is immedeately ready for action again.

Using the Leonardo as USB to serial converter.

Arduino Leonardo used as usb to serial converter.

The green board in the picture above carries an atmega 328p with a Duemilanove bootloader. The Leonardo works as an USB to serial converter and is used to upload sketches to the atmega328p, and to communicate serially with the sketches.

For serial communications and power, the green board has the common 6 pin header you also find on e.g. the Arduino Pro or the Sanguino:

Target processor      Pin                           Serial to usb converter
(Atmega 328p)         Header                        (Leonardo)
GND   ---------------|1|--------------------------- GND
                     |2|                           
5V    <--------------|3|--------------------------- 5V
RX    <--- 1K -------|4|--------------------------- TX
TX    ---- 1K -------|5|--------------------------> RX
RESET <----||--------|6|--------------------------- DTR_PIN (here pin 13)
  |       100nF    
 10K
  |
 5V

The DTR pin is used by the Arduino IDE to automatically trigger a reset of the target processor, when it uploads a sketch. The header is meant to be used with an usb to serial converter like e.g. Sparkfun’s  “FTDI simple break out board”. This post is about a sketch and a few hacks in the arduino core that allow you to use the Leonardo  for this purpose.

Here is the sketch:

/*
  leo_usb2serial
  Allows to use an Arduino Leonardo as an usb to serial converter.
 */
static long baud = 57600;
static long newBaud = baud;

// this pin will output the DTR signal (as set by the pc)
#define DTR_PIN 13

#define LINESTATE_DTR  1

void lineCodingEvent(long baud, byte databits, byte parity, byte charFormat)
{
  newBaud = baud;
}

void lineStateEvent(unsigned char linestate)
{
  if(linestate & LINESTATE_DTR)
    digitalWrite(DTR_PIN, HIGH);
  else
    digitalWrite(DTR_PIN, LOW);
}

void setup() {
  pinMode(DTR_PIN, OUTPUT);
  digitalWrite(DTR_PIN, LOW);
  Serial.begin(baud);
  Serial1.begin(baud);
}

void loop() {

  // Set the new baud rate
  if(newBaud != baud) {
    baud = newBaud;
    Serial1.end();
    Serial1.begin(baud);
  }

  // copy from virtual serial line to uart and vice versa
  if (Serial.available()) {
    char c = (char)Serial.read();
    Serial1.write(c);
  }
  if (Serial1.available()) {
    char c = (char)Serial1.read();
    Serial.write(c);
  }
}

The sketch’s main job is to forward everything that is received over USB (Serial) onto the real uart (Serial1), and vice versa. That is what happens at the end of loop().

Another task the sketch has to accomplish is to update the uart’s baudrate, whenever the pc changes the baud rate of the virtual com port. When this happens, the arduino core calls lineCodingEvent(baud,...). This routine runs under interrupt so we must take care not to spend too much time in it. Therefore the new baud rate is recorded in newBaud and the actual work is done from loop():

 if(newBaud != baud) {
    baud = newBaud;
    Serial1.end();
    Serial1.begin(baud);
  }

The last thing is to make the DTR signal available on one of the digital pins. Whenever the pc sets or clears DTR of the virtual com port, the arduino core calls lineStateEvent(). This routine also runs under interrupt, but the only thing to do is to adjust the level of the DTR_PIN. Pin 13 is used as DTR_PIN, this way one can observe the led to see what the Arduino IDE does with the DTR signal.

Now we get to the hacking part. Neither lineCodingEvent() nor lineStateEvent() are part of the arduino core. I plan to submit a change request for this, it looks a useful feature to me. However, it is not intrusive to add this feature manually to the core. Besides the changes discussed in my previous post are also needed for this to work, otherwise serial buffer overruns will happen.

Locate the CDC.cpp file and add the lines printed in bold:

void WEAK lineCodingEvent(long baud, byte databits, byte parity, byte charFormat)
{
}

void WEAK lineStateEvent(byte linestate)
{
}

bool WEAK CDC_Setup(Setup& setup)
{
        u8 r = setup.bRequest;
        u8 requestType = setup.bmRequestType;

        if (REQUEST_DEVICETOHOST_CLASS_INTERFACE == requestType)
        {
                if (CDC_GET_LINE_CODING == r)
                {
                        USB_SendControl(0,(void*)&_usbLineInfo,7);
                        return true;
                }
        }

        if (REQUEST_HOSTTODEVICE_CLASS_INTERFACE == requestType)
        {
                if (CDC_SET_LINE_CODING == r)
                {
                        USB_RecvControl((void*)&_usbLineInfo,7);
                        lineCodingEvent(_usbLineInfo.dwDTERate,
                                        _usbLineInfo.bDataBits,
                                        _usbLineInfo.bParityType,
                                        _usbLineInfo.bCharFormat);
                        return true;
                }

                if (CDC_SET_CONTROL_LINE_STATE == r)
                {
                        _usbLineInfo.lineState = setup.wValueL;

                        lineStateEvent(_usbLineInfo.lineState);

                        // auto-reset into the bootloader is triggered when the port, already 
                        // open at 1200 bps, is closed.  this is the signal to start the watchdog
                        // with a relatively long period so it can finish housekeeping tasks
                        // like servicing endpoints before the sketch ends
                        if (1200 == _usbLineInfo.dwDTERate) {
                                // We check DTR state to determine if host port is open (bit 0 of lineState).
                                if ((_usbLineInfo.lineState & 0x01) == 0) {
                                        *(uint16_t *)0x0800 = 0x7777;
                                        wdt_enable(WDTO_120MS);
                                } else {
                                        // Most OSs do some intermediate steps when configuring ports and DTR can
                                        // twiggle more than once before stabilizing.
                                        // To avoid spurious resets we set the watchdog to 250ms and eventually
                                        ...

First, weak symbols are add for lineCodingEvent() and lineStateEvent(). This makes sure that if a sketch does not provide one of the functions, the linker will use these dummy ones.

Then, it suffices to call the functions at the right place. CDC_Setup() is called whenever the Leonardo receives a “control message” , related to the CDC (Communication Device Class) protocol. Line coding and line state are set via such control messages. In the above snippet, the event functions are called after the messages are parsed.

I tried out the sketch by downloading the asciiTable sample using the IDE. The target atmega328p gets autoreset and the baudrate updated to 57600, which is what the Duemilanove bootloader uses. When the download completed, I opened the serial monitor and set it to 9600 baud. Upon reset of the atmega328p, I received the expected output from the asciiTable sketch.

ArduinoISP on the Leonardo

A Leonardo programming an attiny2313

A while ago I noticed this thread about burning an Atmega 328 bootloader with an Arduino Leonardo. It turned out that the ArduinoISP sketch that comes with the arduino ide (version 1.0.1) does not work out of the box on the recent Leonardo model. I decided to buy one and help investigating the problems reported in that thread. Following issues were posted:

In mean time fixes for these issues are committed in the development tree so I expect them to become available in an upcoming release of the ide. Untill then, here is a wrap up on how to use the leonardo as an isp programmer with Arduino 1.0.1. Edit: the fix proposed below in step 3 made it into Arduino 1.0.2, so it is no longer needed. The rest is still applicable.

1. Like in the picture above, connect the target’s MISO/MOSI and SCK signals to the ICSP header of the Leonardo. Don’t connect them to digital pins 11, 12 and 13: on the leo there is no SPI on these pins.

Edit: the picture above is just for setting the scene. Ja450n (thank you) sent me a nice Fritzing drawing, featuring an attiny85,  that is more illustrative:
leo_attiny85

2. In ArduinoISP.ino, change this line:
#define RESET SS
into:
#define RESET 10

This states we want to use pin 10, to reset the target mcu .(the brown wire in the picture).
On the leo, SS is not available on arduino pin 10. Actually it is not available on any arduino pin, the atmega pin that exposes SS is used to drive the RX led. Fortunately it is perfectly ok to use pin 10 (or any other digital pin) to drive the target’s reset. (The only requirement for the SS pin is that it is configured as an output (which makes the leo an SPI master). This requirement is fulfilled since it drives the rx led.

3. Locate the CDC.cpp and USBCore.cpp files in your arduino installation directory and apply following modifications (you may want to make a copy first):
In CDC.cpp, the accept routine should look like this:

void Serial_::accept(void)
{
    ring_buffer *buffer = &cdc_rx_buffer;

    int i = (unsigned int)(buffer->head+1) % SERIAL_BUFFER_SIZE;

    // if we should be storing the received character into the location
    // just before the tail (meaning that the head would advance to the
    // current location of the tail), we're about to overflow the buffer
    // and so we don't write the character or advance the head.

    // while we have room to store a byte
    while (i != buffer->tail) {
        int c = USB_Recv(CDC_RX);
        if (c == -1)
            break; // no more data, we're done
        buffer->buffer[buffer->head] = c;
        buffer->head = i;

        i = (unsigned int)(buffer->head+1) % SERIAL_BUFFER_SIZE;
    }
}

In USBCore.cpp, search following snippet:

#ifdef CDC_ENABLED
    USB_Flush(CDC_TX); // Send a tx frame if found
    while (USB_Available(CDC_RX)) // Handle received bytes (if any)
        Serial.accept();
#endif

and change it into:

#ifdef CDC_ENABLED
    USB_Flush(CDC_TX); // Send a tx frame if found
    if (USB_Available(CDC_RX)) // Handle received bytes (if any)
        Serial.accept();
#endif

4. Compile and upload ArduinoISP to the leo.

5. In the ide, open the serial monitor and see to it that the baud rate is not equal to 1200bps. Close the serial monitor again. This step removes the magic baud rate from the serial port which would cause unintentional auto resets.

6. On linux (and probably mac, though I don’t have a mac to try it) you can just go to the “Tools > Programmer” menu and select “Arduino as ISP” as programmer, Now the leo is ready to act as an isp. From here, follow the regular procedure to burn a bootloader in the target. Or you can use avrdude from the command line to burn any hex file in the target…

7. On windows an extra step is required. When selecting “Arduino as ISP” as programmer, the IDE will invoke avrdude and instruct it to use the stk500v1 protocol. Currently (arduino 1.0.2 and 1.0.3) this does not work on windows. It will result in following error message:

avrdude.exe: Send: 0 [30]   [20]
avrdude.exe: Send: 0 [30]   [20]
avrdude.exe: Send: 0 [30]   [20]
avrdude.exe: Recv:
avrdude.exe: stk500_getsync(): not in sync: resp=0x00

Note that avrdude sends out stuff, but does not receive anything. The leo’s rx led will flash but the tx led stays dark.

As a  temporary work around it was suggested (see here) to define a programmer that instructs avrdude to use the “arduino” protocol instead of “stk500v1″. To do so, create a sub directory named “hardware/leofix” in your sketchbook directory. In that directory install a file “programmers.txt” with following content:

arduinoispleo.name=Arduino as ISP (Leonardo)
arduinoispleo.communication=serial
arduinoispleo.protocol=arduino
arduinoispleo.speed=19200

After restarting the IDE, you will have an entry “Arduino as ISP (Leonardo)” in the “Tools > Programmers” menu. This will do the job on windows too.

The reason stk500v1 does not work is that upon opening the virtual com port, windows/avrdude do not request to assert the “virtual” DTR and RTS signals. The leonardo refuses to send out data under these conditions so it never writes data back to avrdude. When using the arduino protocol, before starting programming, avrdude will briefly deassert DTR and RTS and assert them again. This would trigger autoreset if you were uploading a sketch to an arduino bootloader. But in this work around, we use the side effect that this leaves DTR/RTS asserted during programming which makes the leonardo talk..

I filed an issue for this too:

8. Another reason why ArduinoISP may not work, may come from the fact that the Leonardo is a composite device (it combines a virtual comm port and a HID (keyboard/mouse) in one USB device). Obviously, if you are on an old operating system that does not support composite devices, ArduinoISP will not work, nor will any sketch that uses USB. (Confusingly, uploading sketches is possible because the boot loader is not a composite device.)

Windows has support from XP SP3 on and from Vista SP1 on, Mac has support since Lion, see here for more background.

Since ArduinoISP does not need the HID stuff, a work around is to build it as a “serial port only” sketch. This is described here.