OTA upgrade bug, when file not found on server ?

[taribo]

Hi,

I am using the SDK 1.1.2 functions for FOTA (Firmware upgrade over-the-air) such as:
system_upgrade_start(), system_upgrade_userbin_check(), system_upgrade_reboot().

I am using my own OTA server, that holds user1.bin and user2.bin.

When these 2 files (user1.bin and user2.bin) are present on the server, the OTA upgrade works fine - the upgrade is successfully done.

But I have tried the following test:
- the server is up and running
- the 2 files (user1.bin and user2.bin) are NOT present on the server

Normally, I would expect the OTA upgrade to fail, because the server returns file not found when the ESP module tries to download user1.bin or user2.bin.
Instead, the upgrade_flag (in the upgrade_server_info structure) is true. This means the upgrade was a success - but this is an error, because the OTA upgrade failed, as the user1.bin/user2.bin file was not found on the server.

This is the use case:
- I am currently running user1.bin on ESP module - this is FW ver. 2
- on the flash memory, on user2.bin part, I have an older version: FW ver. 1
- on the server, the files user1.bin and user2.bin are NOT present. But the server is up and running
- I begin the OTA upgrade (with system_upgrade_start())
- ESP module send serial debug info - upgrade is successfully done
- ESP module reboots (with system_upgrade_reboot())
- I am now running user2.bin - but this is the older FW ver. 1

So, the bug seems to be in the system_upgrade_start() function - the flag upgrade_flag is set even if the download of the user1.bin/user2.bin file has failed, due to the absence of the file on server.

[ESP_Faye]

Hi，

Sorry that we can't duplicate your problem.

Could you provide your test code ？We will test it.

[taribo]

Hi,
The code I am using is the AT example (\esp_iot_sdk_v1.2.0_15_07_03\esp_iot_sdk_v1.2.0\examples\at). I have tried with SDK v1.2.0 also and the result is the same as in my initial post, with SDK v1.1.2.

I have done only 2 modifications to AT example code, in the file at_upgrade.c:
1. The IP of the server (for me is: host_ip.addr = ipaddr_addr("192.168.1.11");)
2. I added: espconn_disconnect(pespconn); at the end of the function at_upDate_recv(void *arg, char *pusrdata, unsigned short len) - this is needed as my server doesn't know how to disconnect the ESP module.

(Below is the content of the at_upgrade.c, even if it says "user_main.c" in the first comment. this is just a copy-paste mistake...)

Code: Select all

/******************************************************************************
 * Copyright 2015-2018 Espressif Systems (Wuxi)
 *
 * FileName: user_main.c
 *
 * Description: entry file of user application
 *
 * Modification history:
 *     2015/3/06, v1.0 create this file.
*******************************************************************************/
#include "c_types.h"
#include "user_interface.h"
#include "espconn.h"
#include "mem.h"
#include "osapi.h"
#include "upgrade.h"

#ifdef AT_UPGRADE_SUPPORT
#ifdef AT_CUSTOM_UPGRADE

#define UPGRADE_FRAME  "{\"path\": \"/v1/messages/\", \"method\": \"POST\", \"meta\": {\"Authorization\": \"token %s\"},\
\"get\":{\"action\":\"%s\"},\"body\":{\"pre_rom_version\":\"%s\",\"rom_version\":\"%s\"}}\n"

#define pheadbuffer "Connection: keep-alive\r\n\
Cache-Control: no-cache\r\n\
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 \r\n\
Accept: */*\r\n\
Accept-Encoding: gzip,deflate\r\n\
Accept-Language: zh-CN,eb-US;q=0.8\r\n\r\n"

/**/


struct espconn *pespconn = NULL;
struct upgrade_server_info *upServer = NULL;

static os_timer_t at_delay_check;
static struct espconn *pTcpServer = NULL;
static ip_addr_t host_ip;
/******************************************************************************
 * FunctionName : user_esp_platform_upgrade_cb
 * Description  : Processing the downloaded data from the server
 * Parameters   : pespconn -- the espconn used to connetion with the host
 * Returns      : none
*******************************************************************************/
LOCAL void ICACHE_FLASH_ATTR
at_upDate_rsp(void *arg)
{
  struct upgrade_server_info *server = arg;


  if(server->upgrade_flag == true)
  {
    os_printf("device_upgrade_success\r\n");
    at_response_ok();
    system_upgrade_reboot();
  }
  else
  {
    os_printf("device_upgrade_failed\r\n");
    at_response_error();
  }

  os_free(server->url);
  server->url = NULL;
  os_free(server);
  server = NULL;
}
/**
  * @brief  Tcp client disconnect success callback function.
  * @param  arg: contain the ip link information
  * @retval None
  */
static void ICACHE_FLASH_ATTR
at_upDate_discon_cb(void *arg)
{
  struct espconn *pespconn = (struct espconn *)arg;
  uint8_t idTemp = 0;

  if(pespconn->proto.tcp != NULL)
  {
    os_free(pespconn->proto.tcp);
  }
  if(pespconn != NULL)
  {
    os_free(pespconn);
  }

  os_printf("disconnect\r\n");

  if(system_upgrade_start(upServer) == false)
  {
    at_response_error();
  }
  else
  {
    at_port_print("+CIPUPDATE:4\r\n");
  }
}

/**
  * @brief  Udp server receive data callback function.
  * @param  arg: contain the ip link information
  * @retval None
  */
LOCAL void ICACHE_FLASH_ATTR
at_upDate_recv(void *arg, char *pusrdata, unsigned short len)
{
  struct espconn *pespconn = (struct espconn *)arg;
  char temp[32] = {0};
  uint8_t user_bin[12] = {0};
  uint8_t i = 0;

  os_timer_disarm(&at_delay_check);
  at_port_print("+CIPUPDATE:3\r\n");

  upServer = (struct upgrade_server_info *)os_zalloc(sizeof(struct upgrade_server_info));

  upServer->upgrade_version[5] = '\0';

  upServer->pespconn = pespconn;

  os_memcpy(upServer->ip, pespconn->proto.tcp->remote_ip, 4);

  upServer->port = pespconn->proto.tcp->remote_port;

  upServer->check_cb = at_upDate_rsp;
  upServer->check_times = 60000;

  if(upServer->url == NULL)
  {
    upServer->url = (uint8 *) os_zalloc(1024);
  }

  if(system_upgrade_userbin_check() == UPGRADE_FW_BIN1)
  {
    os_memcpy(user_bin, "user2.bin", 10);
  }
  else if(system_upgrade_userbin_check() == UPGRADE_FW_BIN2)
  {
    os_memcpy(user_bin, "user1.bin", 10);
  }

  os_sprintf(upServer->url,
        "GET /%s HTTP/1.1\r\nHost: "IPSTR"\r\n"pheadbuffer"",
        user_bin, IP2STR(upServer->ip));

   espconn_disconnect(pespconn);
}

LOCAL void ICACHE_FLASH_ATTR
at_upDate_wait(void *arg)
{
  struct espconn *pespconn = arg;
  os_timer_disarm(&at_delay_check);
  if(pespconn != NULL)
  {
    espconn_disconnect(pespconn);
  }
  else
  {
    at_response_error();
  }
}

/******************************************************************************
 * FunctionName : user_esp_platform_sent_cb
 * Description  : Data has been sent successfully and acknowledged by the remote host.
 * Parameters   : arg -- Additional argument to pass to the callback function
 * Returns      : none
*******************************************************************************/
LOCAL void ICACHE_FLASH_ATTR
at_upDate_sent_cb(void *arg)
{
  struct espconn *pespconn = arg;
  os_timer_disarm(&at_delay_check);
  os_timer_setfn(&at_delay_check, (os_timer_func_t *)at_upDate_wait, pespconn);
  os_timer_arm(&at_delay_check, 5000, 0);
  os_printf("at_upDate_sent_cb\r\n");
}

/**
  * @brief  Tcp client connect success callback function.
  * @param  arg: contain the ip link information
  * @retval None
  */
static void ICACHE_FLASH_ATTR
at_upDate_connect_cb(void *arg)
{
  struct espconn *pespconn = (struct espconn *)arg;
  uint8_t user_bin[9] = {0};
  char *temp = NULL;

  at_port_print("+CIPUPDATE:2\r\n");


  espconn_regist_disconcb(pespconn, at_upDate_discon_cb);
  espconn_regist_recvcb(pespconn, at_upDate_recv);////////
  espconn_regist_sentcb(pespconn, at_upDate_sent_cb);

  temp = (uint8 *) os_zalloc(512);

  os_sprintf(temp,"GET /v1/device/rom/?is_format_simple=true HTTP/1.0\r\nHost: "IPSTR"\r\n"pheadbuffer"",
             IP2STR(pespconn->proto.tcp->remote_ip));

  espconn_sent(pespconn, temp, os_strlen(temp));
  os_free(temp);
}

/**
  * @brief  Tcp client connect repeat callback function.
  * @param  arg: contain the ip link information
  * @retval None
  */
static void ICACHE_FLASH_ATTR
at_upDate_recon_cb(void *arg, sint8 errType)
{
  struct espconn *pespconn = (struct espconn *)arg;

    at_response_error();
    if(pespconn->proto.tcp != NULL)
    {
      os_free(pespconn->proto.tcp);
    }
    os_free(pespconn);
    os_printf("disconnect\r\n");

    if(upServer != NULL)
    {
      os_free(upServer);
      upServer = NULL;
    }
    at_response_error();

}

/******************************************************************************
 * FunctionName : upServer_dns_found
 * Description  : dns found callback
 * Parameters   : name -- pointer to the name that was looked up.
 *                ipaddr -- pointer to an ip_addr_t containing the IP address of
 *                the hostname, or NULL if the name could not be found (or on any
 *                other error).
 *                callback_arg -- a user-specified callback argument passed to
 *                dns_gethostbyname
 * Returns      : none
*******************************************************************************/
LOCAL void ICACHE_FLASH_ATTR
upServer_dns_found(const char *name, ip_addr_t *ipaddr, void *arg)
{
  struct espconn *pespconn = (struct espconn *) arg;
//  char temp[32];

  if(ipaddr == NULL)
  {
    at_response_error();
    return;
  }
  at_port_print("+CIPUPDATE:1\r\n");


  if(host_ip.addr == 0 && ipaddr->addr != 0)
  {
    if(pespconn->type == ESPCONN_TCP)
    {
      os_memcpy(pespconn->proto.tcp->remote_ip, &ipaddr->addr, 4);
      espconn_regist_connectcb(pespconn, at_upDate_connect_cb);
      espconn_regist_reconcb(pespconn, at_upDate_recon_cb);
      espconn_connect(pespconn);
    }
  }
}

void ICACHE_FLASH_ATTR
at_exeCmdCiupdate(uint8_t id)
{
  pespconn = (struct espconn *)os_zalloc(sizeof(struct espconn));
  pespconn->type = ESPCONN_TCP;
  pespconn->state = ESPCONN_NONE;
  pespconn->proto.tcp = (esp_tcp *)os_zalloc(sizeof(esp_tcp));
  pespconn->proto.tcp->local_port = espconn_port();
  pespconn->proto.tcp->remote_port = 80;

  host_ip.addr = ipaddr_addr("192.168.1.11");
  at_port_print("+CIPUPDATE:1\r\n");
  os_memcpy(pespconn->proto.tcp->remote_ip, &host_ip.addr, 4);
  espconn_regist_connectcb(pespconn, at_upDate_connect_cb);
  espconn_regist_reconcb(pespconn, at_upDate_recon_cb);
  espconn_connect(pespconn);
}
#endif
#endif


I have a 32Mbit flash, so at step 5 of the make, I choose 4=4096KB(512KB+512KB)

Using ESP FLASH DOWNLOAD TOOL V1.2, I wrote to flash successfully:
- boot_v1.4(b1).bin -> 0x00000
- user1.4096.new.4.bin -> 0x01000
- esp_init_data_default.bin -> 0x3FC000
- blank.bin -> 0x3FE000

Now I have user1.bin in flash. At the address 0x81000 there is nothing (here we need to write user2.bin).

I have started my web server - but the files user1.bin and user2.bin are NOT present on this server.

On the serial port, I give the following commands:

Code: Select all

23:21:17.703> AT+RST
23:21:17.703>
23:21:17.703>
23:21:17.703> OK
23:21:17.703> WIFI DISCONNECT
23:21:17.828>
23:21:17.828>  ets Jan  8 2013,rst cause:4, boot mode:(3,7)
23:21:17.828>
23:21:17.828> wdt reset
23:21:17.906> load 0x40100000, len 1396, room 16
23:21:17.906> tail 4
23:21:17.906> chksum 0x89
23:21:17.906> load 0x3ffe8000, len 776, room 4
23:21:17.906> tail 4
23:21:17.906> chksum 0xe8
23:21:17.906> load 0x3ffe8308, len 540, room 4
23:21:17.906> tail 8
23:21:17.906> chksum 0xc0
23:21:17.906> csum 0xc0
23:21:17.906>
23:21:17.906> 2nd boot version : 1.4(b1)
23:21:17.906>   SPI Speed      : 40MHz
23:21:17.906>   SPI Mode       : QIO
23:21:17.906>   SPI Flash Size & Map: 32Mbit(512KB+512KB)
23:21:17.906> jump to run user1 @ 1000
23:21:17.906>
23:21:18.078> rlŽ‚rô
23:21:18.078> ready
23:21:21.140> WIFI CONNECTED
23:21:24.015> WIFI GOT IP
23:21:26.515> AT+GMR
23:21:26.515>
23:21:26.515> AT version:0.30.0.0(Jul  3 2015 19:35:49)
23:21:26.515> SDK version:1.2.0
23:21:26.515> compile time:Jul 14 2015 22:25:19
23:21:26.515> OK
23:21:34.828> AT+CIFSR
23:21:34.828>
23:21:34.828> +CIFSR:APIP,"192.168.4.1"
23:21:34.828> +CIFSR:APMAC,"1a:fe:34:xx:xx:xx"
23:21:34.828> +CIFSR:STAIP,"192.168.1.171"
23:21:34.828> +CIFSR:STAMAC,"18:fe:34:xx:xx:xx"
23:21:34.828>
23:21:34.828> OK
23:21:40.015> AT+CIUPDATE
23:21:40.015>
23:21:40.015> +CIPUPDATE:1
23:21:40.203> +CIPUPDATE:2
23:21:40.203> +CIPUPDATE:3
23:21:40.203> +CIPUPDATE:4
23:21:40.203>
23:21:40.203> OK
23:21:40.265> WIFI DISCONNECT
23:21:40.390>
23:21:40.390>  ets Jan  8 2013,rst cause:4, boot mode:(3,7)
23:21:40.390>
23:21:40.390> wdt reset
23:21:40.453> load 0x40100000, len 1396, room 16
23:21:40.453> tail 4
23:21:40.453> chksum 0x89
23:21:40.453> load 0x3ffe8000, len 776, room 4
23:21:40.453> tail 4
23:21:40.453> chksum 0xe8
23:21:40.453> load 0x3ffe8308, len 540, room 4
23:21:40.453> tail 8
23:21:40.453> chksum 0xc0
23:21:40.453> csum 0xc0
23:21:40.453>
23:21:40.453> 2nd boot version : 1.4(b1)
23:21:40.453>   SPI Speed      : 40MHz
23:21:40.453>   SPI Mode       : QIO
23:21:40.453>   SPI Flash Size & Map: 32Mbit(512KB+512KB)
23:21:40.453> jump to run user2 @ 81000
23:21:40.453>
23:21:40.453> error magic!
23:21:40.640> first boot failed, reboot to try backup bin
23:21:40.640>
23:21:41.703>
23:21:41.703>  ets Jan  8 2013,rst cause:4, boot mode:(3,7)
23:21:41.703>
23:21:41.703> wdt reset
23:21:41.703> load 0x40100000, len 1396, room 16
23:21:41.812> tail 4
23:21:41.812> chksum 0x89
23:21:41.812> load 0x3ffe8000, len 776, room 4
23:21:41.812> tail 4
23:21:41.812> chksum 0xe8
23:21:41.812> load 0x3ffe8308, len 540, room 4
23:21:41.812> tail 8
23:21:41.812> chksum 0xc0
23:21:41.812> csum 0xc0
23:21:41.812>
23:21:41.812> 2nd boot version : 1.4(b1)
23:21:41.812>   SPI Speed      : 40MHz
23:21:41.812>   SPI Mode       : QIO
23:21:41.812>   SPI Flash Size & Map: 32Mbit(512KB+512KB)
23:21:41.812> jump to run user1 @ 1000
23:21:41.812>
23:21:41.937> rlŽ‚rô
23:21:41.937> ready
23:21:45.015> WIFI CONNECTED
23:21:47.890> WIFI GOT IP


As you can see from the AT commands, after AT+CIUPDATE command, the ESP module responds with +CIPUPDATE:1-4, then OK. This means the update was successfully done.
But there is no user1.bin or user2.bin on my server, so there is a bug here.

After +CIPUPDATE:1-4, OK, I see the module reboots. It tries to "jump to run user2 @ 81000" but there is no code here - thankfully is able to see that is no code and it says: "error magic!first boot failed, reboot to try backup bin" then reboots in user1.

In another use case, when I actualy have a user2.bin written in flash @ 81000 (a previous version of FW), when I try the above procedure, after +CIPUPDATE:1-4, OK, it jumps to user2.bin and runs a previous version of FW.

Please let me know if you need any more details.

[ESP_Faye]

Hi,

Sorry that we still can't duplicate your problem..

Here is an example of AT upgrade http://bbs.espressif.com/viewtopic.php?f=21&t=732, could you have a try with tool "MiniWebServer" there ? It is Chinese version ..

Could you zip and upload your webserver here ? We will have a try.

Could you capture packets with "wireshark" while upgrading ?

Sorry for the inconvenience.

[taribo]

Hi,

As a server, I am using the IIS7 default server in Win7 Pro.

Please find attached the wireshark capture export, while upgrading.
192.168.1.171 is the IP of the ESP module. 192.168.1.11 is the IP of the server. As you can see from the capture, the file user2.bin is not found on server. But still, on ESP serial port I receive +CIPUPDATE:1-4, then OK, then reboot.

I have tried the application "MiniWebServer" you recommended. With this server application, I didn't encountered any problems. When user1.bin and user2.bin are not present, in this server application, the ESP module gives me an error while upgrading (as it should, because the upgrade is NOT done).

[ESP_Faye]

Hi，

We optimized OTA function.

Please have a try with the attachment，based on SDK_v1.2.0

Thanks for your interest in Espressif Systems and ESP8266 !

[taribo]

Hi,

I made a quick test with provided libupgrade.a, on my IIS7 server and indeed, the bug is fixed.
Now I get an error when the user1.bin/user2.bin are not present on my server - this is the normal behavior.

I will test later with different servers and get back only if I see any strange behavior.

PS: Am I eligible for Bug Bounty Program with this bug ?

[ESP_Faye]

Hi，

So sorry that this report is not eligible for Bug Bounty Program.

If you want to win a Bug Bounty, please don't report it on BBS, report it following this http://bbs.espressif.com/viewtopic.php?f=5&t=288

Thanks for your interest in ESP8266 !