ESP8266 Developer Zone

Re: Memory Leak/Usage

2017-03-02T22:38:37+08:00

Hi All,

We are using ESP RTOS SDK 1.5.0 and ported OpenSSL from espressif/esp8266-rtos-sample-code from Github.

Both OpenSSL Client and Server programs executed successfully but consumes almost 30 KB to 33 KB of Heap Memory. Once we free SSL then we can get original memory which are available when we started example.

Also, There are 2 static library OpenSSL and EmbedTLS Libraries are used to execute OpenSSL Client/Server Program.

Doe anyone has any idea for why it consumes that much of memory? If possible then provide any other low memory consume library for secure connection like OpenSSL.

Statistics: Posted by ritesh — Thu Mar 02, 2017 10:38 pm

Re: Memory Leak/Usage

2016-11-03T20:54:11+08:00

Hi,
I'm having the same problem as OP.

I'm running the "OS SDK ver: 1.4.0(c599790) compiled @ Mar 1 2016 17:25:17" sdk version.

No matter where I put the

Code:

espconn_set_opt(connPtr, ESPCONN_REUSEADDR);

there's always some pending bytes that don't get freed after disconnect.

Code:

system_get_free_heap_size: 9632.
Client connected.
Request received.
Response sent.
Client disconnected.
system_get_free_heap_size: 9408.

Is there a patch needed for RTOS as well?

Statistics: Posted by gustavo — Thu Nov 03, 2016 8:54 pm

Re: Memory Leak/Usage

2015-09-10T16:24:43+08:00

Hi,

Sorry that we only support ROOT CA, do not support Intermediate CA + root CA right now.

But your request is very important to us, we will think over it and support it in the future.

Thanks for your interest in ESP8266 !

Statistics: Posted by ESP_Faye — Thu Sep 10, 2015 4:24 pm

Re: Memory Leak/Usage

2015-08-28T13:23:24+08:00

Hi, Faye!

Espressif_Faye wrote:
Hi,

Is the "https://ssl.test.com" your SSL server ? We can not access to it by browser.

What is the IP, port and CA cert of your SSL server?

No, i've send you the real server and both .bin files in a private message.
I see it is in my outbox and not in the send items.
Please, take a look.

Statistics: Posted by zaxl — Fri Aug 28, 2015 1:23 pm

Re: Memory Leak/Usage

2015-08-28T12:44:38+08:00

Hi,

Is the "https://ssl.test.com" your SSL server ? We can not access to it by browser.

What is the IP, port and CA cert of your SSL server?

Statistics: Posted by ESP_Faye — Fri Aug 28, 2015 12:44 pm

Re: Memory Leak/Usage

2015-08-27T20:54:04+08:00

Hi, Faye!

Espressif_Faye wrote:
Hi,

Is the "https://ssl.test.com" your SSL server ? We can not access to it by browser.

What is the IP, port and CA cert of your SSL server?

Check your private messages about my server and ca cert details.

Code:
espconn_secure_ca_enable(ESPCONN_CLIENT, CA_FLASH_ADDR); // flash sector which stored "esp_ca_cert.bin" espconn_secure_cert_req_enable(ESPCONN_CLIENT, CERT_FLASH_ADDR); // flash sector which stored "esp_cert_private_key.bin" espconn_secure_connect(&conn);

How to get "esp_ca_cert.bin", please refer to documentation "5A-ESP8266__SDK__SSL_User_Manual" in \esp_iot_sdk_v1.3.0\document, tools in \esp_iot_sdk_v1.3.0\tools.

How to get "esp_cert_private_key.bin", please refer to the attachment.

Having two scripts called make_cert.py is a bit confusing.
When acting as a client it doesn't make sense to have all the certs and keys issued merged -
so i used a simple
cat private.key certificate.key > esp_cert_private_key.bin
the script does exactly the same.
{ code }
int sect;
sect = 0x6b;
os_printf("cachain at %x\n", sect);
espconn_secure_ca_enable(ESPCONN_CLIENT, sect);
sect = 0x6c;
os_printf("client key at %x\n", sect);
espconn_secure_cert_req_enable(ESPCONN_CLIENT, sect);

{ end code }

cachain at 6b
client key at 6c
E:M 51440
Fatal exception (28):
epc1=0x4000e1e0, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000
/ 0x4000e1e0 is at pm_set_sleep_time but i'm not sure if i got it right /

Statistics: Posted by zaxl — Thu Aug 27, 2015 8:54 pm

Re: Memory Leak/Usage

2015-08-26T17:31:05+08:00

Hi,

Is the "https://ssl.test.com" your SSL server ? We can not access to it by browser.

What is the IP, port and CA cert of your SSL server?

Code:

espconn_secure_ca_enable(ESPCONN_CLIENT, CA_FLASH_ADDR); // flash sector which stored "esp_ca_cert.bin"
espconn_secure_cert_req_enable(ESPCONN_CLIENT, CERT_FLASH_ADDR); // flash sector which stored "esp_cert_private_key.bin"
espconn_secure_connect(&conn);

How to get "esp_ca_cert.bin", please refer to documentation "5A-ESP8266__SDK__SSL_User_Manual" in \esp_iot_sdk_v1.3.0\document, tools in \esp_iot_sdk_v1.3.0\tools.

How to get "esp_cert_private_key.bin", please refer to the attachment.

Certificate Request.zip

Statistics: Posted by ESP_Faye — Wed Aug 26, 2015 5:31 pm

Re: Memory Leak/Usage

2015-08-25T20:29:15+08:00

Hi, Faye!

Espressif_Faye wrote:
Hi,

There are two new APIs in the new lib that may help, please have a try.
API1.jpg

API2.jpg

If i got it right - tested with:
espconn_secure_cert_req_disable(3); // disable client certificates in server and client

The result is the same as the previous test.

I do not use client certificates /but it's a very welcome addition!/ - i want to verify that the server i'm connected to is authentic.
May be an option to consider self signed certificates valid would help in my case.

Statistics: Posted by zaxl — Tue Aug 25, 2015 8:29 pm

Re: Memory Leak/Usage

2015-08-25T20:05:56+08:00

Hi,

There are two new APIs in the new lib that may help, please have a try.

Statistics: Posted by ESP_Faye — Tue Aug 25, 2015 8:05 pm

Re: Memory Leak/Usage

2015-08-24T14:49:21+08:00

Hi, Faye!

Espressif_Faye wrote:
Hi,

Please have a try with this new libssl, based on esp_iot_sdk_v1.3.0

I'm sorry but it didn't work.
Test with only the server cert - OK
Test with server and issuer cert - Fail

But there is a little improvement - now it fails immediately - does not wait for server close.

Statistics: Posted by zaxl — Mon Aug 24, 2015 2:49 pm

Re: Memory Leak/Usage

2015-08-24T10:12:30+08:00

Hi,

Please have a try with this new libssl, based on esp_iot_sdk_v1.3.0

libssl.zip

Statistics: Posted by ESP_Faye — Mon Aug 24, 2015 10:12 am

Re: Memory Leak/Usage

2015-08-19T17:43:43+08:00

Hi,

If using v1.3.0 without SSL memory leak patch it does work with iot.espressif.cn.
If using v1.3.0 WITH SSL patch - it does work with stock v1.3.0 but doesn't work with ssl patch.

Works with ssl patch if i call secure_ca_enable after setting the buffer size.
espconn_secure_set_size(ESPCONN_CLIENT,SSLBUF);
secure_ca_enable(1,0x3d);
but if the call chain is in reverse doesn't work.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
the above observations may be messed up since
now i see that you made changes on iot.espressif.cn
openssl s_client -connect iot.espressif.cn:443
Results in:
Verify return code: 20 (unable to get local issuer certificate)

I did the same on my server /removed the issuer certificate from the chain/ and the handshake went ok.
So seems the issue is with the issuer certificate verification.

Statistics: Posted by zaxl — Wed Aug 19, 2015 5:43 pm

Re: Memory Leak/Usage

2015-08-19T16:28:15+08:00

Hi，

Here is my test project and test log, it can connect to Espressif Cloud successfully.

The last log "tcp disconnect succeed !!! " means the Espressif Cloud disconnect to ESP8266, it is not a problem.

Please have a try.

SSL_CA_TEST_LOG.zip

esp_iot_sdk_v1.3.0_BBP175.zip

Statistics: Posted by ESP_Faye — Wed Aug 19, 2015 4:28 pm

Re: Memory Leak/Usage

2015-08-18T14:48:58+08:00

Espressif_Faye wrote:
Hi,

Could you have a try with our SSL server "iot.espressif.cn" port 443 ?

Here is the CA of our SSL server.

Documentation "5A-ESP8266__SDK__SSL_User_Manual" is in \esp_iot_sdk_v1.3.0\document\

Hi,
I've replaced the esp_ca_cert.bin with espressif's and it fails again:
.....
dhcp client start...
chg_B:-60
FH:46632 t:4
Got ip
ip:192.168.0.104,mask:255.255.255.0,gw:192.168.0.1
Starting request to:[https://iot.espressif.cn/]
hostname=iot.espressif.cn
port=443
path=/
DNS request
DNS pending
FH:46360 t:6
DNS found iot.espressif.cn 115.29.202.58
FH:28672 t:7
client handshake start.
client handshake failed
Disconnected
http_status=-1
httperr:-1
FH:44480 t:8
..........

BUT this time faster.
With my self signed certificate - it fails and waits 5 minutes for the server to close..
With espressif's certificate which is not self signed - it disconnects how it should - right after the handshake failure.

Other minor difference to note is that espressif's server accepts client certificates - mine does not.

Statistics: Posted by zaxl — Tue Aug 18, 2015 2:48 pm

Re: Memory Leak/Usage

2015-08-18T14:06:15+08:00

Hi,

Could you have a try with our SSL server "iot.espressif.cn" port 443 ?

Here is the CA of our SSL server.

Documentation "5A-ESP8266__SDK__SSL_User_Manual" is in \esp_iot_sdk_v1.3.0\document\

esp_ca_cert.zip

Statistics: Posted by ESP_Faye — Tue Aug 18, 2015 2:06 pm

Re: Memory Leak/Usage

2015-08-17T18:36:51+08:00

Hi, Faye,

Espressif_Faye wrote:
Hi,

Please call espconn_regist_disconcb before espconn_secure_connect, for example
Code:
espconn_regist_connectcb(pespconn, user_tcp_connect_cb); espconn_regist_reconcb(pespconn, user_tcp_recon_cb); // register disconnect cb here, because that SSL handshake may fail. espconn_regist_disconcb(pespconn, user_tcp_discon_cb); //espconn_secure_set_size(ESPCONN_CLIENT,5120); espconn_secure_connect(pespconn); // tcp SSL connect

If your problem is still unsolved, please provide your test code, we will have a try.

It's there but the disconnect callback is actually called but after 5 minutes , when the server closes the connection.
/see the tcpdump.log/ And not after the handshake failure.

I'm attaching archive with test code, there is a README.txt containing relevant info and instructions.
I'm testing against sdk1.3.0 + ssl patch.

If i do not do secure_ca_enable - everything is fine.

So there are two problems:
1. CA verification does not work
2. Disconnect callback is called on server close , not when the handshake fails

ssltest.tgz

stdout.log.gz

tcpdump.log.txt.gz

Statistics: Posted by zaxl — Mon Aug 17, 2015 6:36 pm

Re: Memory Leak/Usage

2015-08-14T11:27:26+08:00

Hi,

Please call espconn_regist_disconcb before espconn_secure_connect, for example

Code:

        espconn_regist_connectcb(pespconn, user_tcp_connect_cb); 
        espconn_regist_reconcb(pespconn, user_tcp_recon_cb); 
        // register disconnect cb here, because that SSL handshake may fail.
        espconn_regist_disconcb(pespconn, user_tcp_discon_cb);  
        
       //espconn_secure_set_size(ESPCONN_CLIENT,5120); 
       
        espconn_secure_connect(pespconn); // tcp SSL connect

If your problem is still unsolved, please provide your test code, we will have a try.

Statistics: Posted by ESP_Faye — Fri Aug 14, 2015 11:27 am

Re: Memory Leak/Usage

2015-08-13T13:42:32+08:00

Espressif_Faye wrote:
Hi,

Did your espconn_secure_connect return ESPCONN_OK ?

Yes:
esp_secure_connect:0
client handshake start.
client handshake failed

esp_ca_cert.bin size is 1649 bytes.

SSL buffer size is 5120 - i've received errors before when it was to small.
netstat on the server shows the connection in established state.

openssl s_client -connect server:443 reports:

depth=2 C = BG, ST = BG, O = Test, OU = RCA, CN = RCA
verify error:num=19:self signed certificate in certificate chain
verify return:0

Certificate chain
0 s:/C=BG/ST=BG/O=Test/OU=Test/CN=Test
i:/C=BG/ST=BG/O=Test/OU=ICA/CN=ICA
1 s:/C=BG/ST=BG/O=Test/OU=ICA/CN=ICA
i:/C=BG/ST=BG/O=Test/OU=RCA/CN=RCA
2 s:/C=BG/ST=BG/O=Test/OU=RCA/CN=RCA
i:/C=BG/ST=BG/O=Test/OU=RCA/CN=RCA
-- BEGIN CERTIFICATE ---
.....
-----END CERTIFICATE-----
subject=/C=BG/ST=BG/O=Test/OU=Test/CN=Test
issuer=/C=BG/ST=BG/O=Test/OU=ICA/CN=ICA
---
No client certificate CA names sent
---
SSL handshake has read 3075 bytes and written 421 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256

Statistics: Posted by zaxl — Thu Aug 13, 2015 1:42 pm

Re: Memory Leak/Usage

2015-08-13T10:50:29+08:00

Hi,

Did your espconn_secure_connect return ESPCONN_OK ?

Statistics: Posted by ESP_Faye — Thu Aug 13, 2015 10:50 am

Re: Memory Leak/Usage

2015-08-13T04:44:50+08:00

Espressif_Faye wrote:
Hi,

So sorry for the inconvenience.

Please update to esp_iot_sdk_v1.3.0 and try the SSL patch here http://bbs.espressif.com/viewtopic.php?f=46&t=925

Please have a try. If your problem is still unsolved, feel free to let us know !

Thanks for your interest in Espressif Systems and ESP8266 !

Yes, it fixes the missing disconnect callbacks. Leak is gone too - will torture it a little bit more but it's fine. Thanks!!
The SDK 1.3.0 frees some more memory which is great. Keep the good work!

However I've faced the next issue now - the more free memory allows me to enable the ca server verification -
i got just handshake fail debug message - no reconnect or disconnect callback- the connection stays up until the server closes, i.e. no error code. Any pointers on how to debug/resolve this? Web server is apache , i've placed the host as the default vhost so the esp ssl clients get the first certificate during the ssl handshake. I'm using a self signed certificate /CA and Intermediate CA/ - would it pass the verification?

Statistics: Posted by zaxl — Thu Aug 13, 2015 4:44 am

Re: SSL example

2015-08-13T02:37:27+08:00

Thank you Rudi!

any chance to "share" your knowledge with me??

Cheers!

rudi wrote:
hdrut wrote:Hi Rudi yes! That was the problem....

..How are you doing? Have you managed to do some progress on copy protection for ESP?

I am now trying SSL communication, just started...

Cheers, i am going to celebrate my birthday today !!

Hi Horacio ! All the Best from me to you!
Happy Birthday to you!
https://www.youtube.com/watch?v=Vdsqbh_0zzg

Thanks i am fine, hope u too
Yes, copy protect runs like a charme )))
hope for open src the bootloader code from espressif, will be touch more possibles.

SSL works fine, Horacio, 'huafan' has made fine examples too, have a look and a try:
viewtopic.php?f=7&t=386

celebrate your birthday fine! - see you 'jumping' and hear you singing
la la lalalala yeap ...

best wishes!
rudi

edit: see at this too:
viewtopic.php?f=16&p=3173#p3173

Statistics: Posted by hdrut — Thu Aug 13, 2015 2:37 am

Re: SSL example

2015-08-12T21:29:33+08:00

hdrut wrote:
Hi Rudi yes! That was the problem....

..How are you doing? Have you managed to do some progress on copy protection for ESP?

I am now trying SSL communication, just started...

Cheers, i am going to celebrate my birthday today !!

Hi Horacio ! All the Best from me to you!
Happy Birthday to you!
https://www.youtube.com/watch?v=Vdsqbh_0zzg

Thanks i am fine, hope u too
Yes, copy protect runs like a charme )))
hope for open src the bootloader code from espressif, will be touch more possibles.

SSL works fine, Horacio, 'huafan' has made fine examples too, have a look and a try:
viewtopic.php?f=7&t=386

celebrate your birthday fine! - see you 'jumping' and hear you singing
la la lalalala yeap ...

best wishes!
rudi

edit: see at this too:
viewtopic.php?f=16&p=3173#p3173

Statistics: Posted by rudi — Wed Aug 12, 2015 9:29 pm

Re: SSL example

2015-08-12T20:14:56+08:00

Hi Rudi

yes! That was the problem, how silly of me

How are you doing? Have you managed to do some progress on copy protection for ESP?

I am now trying SSL communication, just started...

Cheers, i am going to celebrate my birthday today !!

Hope to hear from you soon, wish all is fine with you Rudi,

Horacio

rudi wrote:
hdrut wrote:Hi,

I'm trying to implement the SSL example provided by Espressif, but get the following errors I can't debug:

undefined reference to `espconn_secure_sent'
undefined reference to `espconn_secure_connect'

Any help is appreciated.

Rgds

think you simple forget "to add" the ssl lib in makefile?

example:

Code:
LIBS = c gcc hal phy pp net80211 lwip wpa upgrade main ssl

write please, how you get on in this.

best wishes
rudi

Statistics: Posted by hdrut — Wed Aug 12, 2015 8:14 pm

Re: SSL example

2015-08-12T15:15:10+08:00

hdrut wrote:
Hi,

I'm trying to implement the SSL example provided by Espressif, but get the following errors I can't debug:

undefined reference to `espconn_secure_sent'
undefined reference to `espconn_secure_connect'

Any help is appreciated.

Rgds

think you simple forget "to add" the ssl lib in makefile?

example:

Code:


LIBS        = c gcc hal phy pp net80211 lwip wpa upgrade main ssl

write please, how you get on in this.

best wishes
rudi

Statistics: Posted by rudi — Wed Aug 12, 2015 3:15 pm

SSL example

2015-08-12T00:33:52+08:00

Hi,

I'm trying to implement the SSL example provided by Espressif, but get the following errors I can't debug:

undefined reference to `espconn_secure_sent'
undefined reference to `espconn_secure_connect'

Any help is appreciated.

Rgds

Statistics: Posted by hdrut — Wed Aug 12, 2015 12:33 am

Re: Memory Leak/Usage

2015-08-11T01:17:09+08:00

The patch fixes the issue and all works as expected, thanks!

Statistics: Posted by mharizanov — Tue Aug 11, 2015 1:17 am

Re: Memory Leak/Usage

2015-08-10T16:33:07+08:00

Hi,

So sorry for the inconvenience.

Please update to esp_iot_sdk_v1.3.0 and try the SSL patch here http://bbs.espressif.com/viewtopic.php?f=46&t=925

Please have a try. If your problem is still unsolved, feel free to let us know !

Thanks for your interest in Espressif Systems and ESP8266 !

Statistics: Posted by ESP_Faye — Mon Aug 10, 2015 4:33 pm

Re: Memory Leak/Usage

2015-08-10T14:28:55+08:00

I can confirm the exact same: I am running MQTT over TLS and esphttpd over non-TLS connection. As of SDK 1.2 the disconnect callback on non-TLS connections (esphttpd) is broken and if there is a SSL connection active (MQTT in my case). As a result, there is a memory leak introduced. If I run the MQTT client over non-TLS, all work just fine and disconnect callback gets called.
I tried with SDK 1.2 and 1.3 and it fails with both. It works just as expected with SDK 1.1

Statistics: Posted by mharizanov — Mon Aug 10, 2015 2:28 pm

Re: Memory Leak/Usage

2015-08-09T08:52:52+08:00

Hi,

Please have a try with the latest sdk_v1.3.0 http://bbs.espressif.com/viewtopic.php?f=46&t=918&p=3092

If your problem is still unsolved, please provide your test code. We will debug it.

Thanks for your interest in ESP8266!

Statistics: Posted by ESP_Faye — Sun Aug 09, 2015 8:52 am

Re: Memory Leak/Usage

2015-08-05T14:01:26+08:00

Hi,

I'm working on a test case code, i'll give it out when ready.
My code flow is like this:
1. listen on port 80 -> web server
2. outbound ssl connection to server. close connection
3. start normal connection to server - long lived

So far i found out that:
If there is only listen on port 80 -> disconnect callback is called - no leak
If there is outbound connection /don't know which one is causing the problem yet/ -> disconnect callback is not called and it leaks.

Statistics: Posted by zaxl — Wed Aug 05, 2015 2:01 pm

Re: Memory Leak/Usage

2015-08-03T10:06:57+08:00

Hi,

So sorry for the inconvenience. Could you provide your test code ? We will have a try.

espconn_regist_reconcb(conn, httpdReconCb);
espconn_regist_disconcb(conn, httpdDisconCb);

If the TCP connection broke unexpected, httpdReconCb will be called.
If espconn_disconnect is called or normally TCP disconnect, it will enter httpdDisconCb.

Statistics: Posted by ESP_Faye — Mon Aug 03, 2015 10:06 am

Memory Leak/Usage

2015-07-23T16:41:55+08:00

Hi,

I'm using the memory optimized SDK 1.2.0.

The issue is that my tcp disconnect callback is not called and each accepted tcp connection leaks between 108 and 288 bytes.
/Will figure exact number but this is from a timer that runs every second to report free heap/.
Tried to set option REUSEADDR but does not make any difference.
I'm using code based on the esp_httpd .
init ....
espconn_regist_connectcb(&httpdConn, httpdConnectCb);
espconn_tcp_set_max_con_allow(&httpdConn, MAX_CONN);
espconn_set_opt(&httpdConn, ESPCONN_REUSEADDR);
espconn_accept(&httpdConn);

static void ICACHE_FLASH_ATTR httpdConnectCb(void *arg) {
....
....
espconn_regist_recvcb(conn, httpdRecvCb);
espconn_regist_reconcb(conn, httpdReconCb);
espconn_regist_disconcb(conn, httpdDisconCb);
espconn_regist_sentcb(conn, httpdSentCb);

}

I'm closing the connection from my main loop but httpdDisconCb is not called.

The incoming connection limit is a kind of useless - with 2 minutes timewait i can generate enough connections to trigger an out of memory condition. Connect/Close 200 bytes per connection about 100 connections and the device is OOM.
What is the way to deal with TCP memory in general?

Regards

Statistics: Posted by zaxl — Thu Jul 23, 2015 4:41 pm