ESP8266 Developer Zone

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-08-29T08:04:51+08:00

Hi @pratik,

Yes the LwIP and mbedTLS sources are public however the ESP link layer code (the /app directory) was never available, and this is what they have now included in their source releases. It was also an issue because it was never clear exactly what version of the open libraries they were using or whether they had made any code changes to the open parts of the source. This is a huge step forward.

I just saw they have now also released sources to the espconn library in the RTOS SDK. This is fantastic news.

Statistics: Posted by davydnorris — Tue Aug 29, 2017 8:04 am

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-08-27T15:43:45+08:00

The sources for LwIP and mbedTLS have always been open and public. You may try to modify the code yourself if you can and share the resultant code if it works. It is really difficult to debug issues on our end because often the connection/disconnection callbacks depend on the routers and when the event happened.
You are responsible for disconnectng all existing connections when WiFi goes out. The LwIP stack can be made to do that, but this is best done in application code itself. Application code can malfunction if the connections just disappeared inside of LwIP in the background.

Statistics: Posted by Guest — Sun Aug 27, 2017 3:43 pm

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-08-12T11:59:36+08:00

Hi @ememberus,

The source for the lwip and mbedtls libraries has now been released in the NONOS SDK on Github:
https://github.com/espressif/ESP8266_NONOS_SDK

Have a look under third_party.

There have also very recently been some fixes and a couple of pull requests addressing leaks and connections bugs in the RTOS Github repo versions of these libraries. They haven't been merged into the NONOS version yet but you could do it yourself:
https://github.com/espressif/ESP8266_RTOS_SDK/commits/master

I am busy with other things but I want to try this myself - if you get a chance please let us know if it solves your problem.

Statistics: Posted by davydnorris — Sat Aug 12, 2017 11:59 am

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-08-10T23:54:38+08:00

Pratik, here are few clarifying notes

- In order to avoid memory leaks, I no longer use secure layer.

- Problem: TCP stack in SDK is unstable by itself - it fails to open
new connections after a number of previous disconnects on WLAN level
(this was previously reported by others).

- Hypothesis: this seems to me like a connection resource leak, when
broken TCP connections are not released back into connection pool.

- Hint: this is likely caused by a race condition, because it is not
100% reproducible, but sooner or later the SDK eventually goes into
this state and becomes dysfunctional.

- Question: you seem to have access to the SDK code.
Could you take a look into this?
I am using ESP8266_NONOS_SDK-2.1.0

Thanks

Statistics: Posted by ememberus — Thu Aug 10, 2017 11:54 pm

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-08-08T15:50:02+08:00

I've never encountered such issues, it would be nice if you could share the code that is causing the problem. All WiFi networking functions will return with error instead of waiting for resource allocation.
You can look at the code for LwIP or mbedTLS (they are open), if that would help. With FreeRTOS, you might want to make sure you are not exceeding the stack size allotted to a task.
Also, do not starve the idle task, because that's the one which handles memory deallocation.

Statistics: Posted by Guest — Tue Aug 08, 2017 3:50 pm

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-27T00:45:53+08:00

You will get a reboot if there is an issue, always.

in the order of importance:

1. in this particular case, the system freezes, but does not reboot,
simply because it is running an internal loop that resets WD timer.

2. the system should never reboot unless the error is fatal.
A "fatal error" means that the system cannot function whatsoever.
Low memory should not be a fatal error.
Give the application some time to complete its current
task so it would release previously allocated memory.

3. system reboot should only be last resort when where is no other
way to recover from an error.

Statistics: Posted by ememberus — Thu Jul 27, 2017 12:45 am

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-27T00:34:58+08:00

Note that no silent action is ever taken up by SDK.

I dare to disagree with this statement, because I personally
encountered these in non-RTOS library.

Endless loops while waiting for resources is a doubtful practice.
Such loops should be interrupted and error generated.

Apart from hiding such internal issues from developers,
lots of other things in SDK are also not documented.
We often have hard time figuring out what went wrong
and how to resolve it.

Of course, we can certainly make and "educated guess", but
do you really think this is how SDK issues should be handled?

Statistics: Posted by ememberus — Thu Jul 27, 2017 12:34 am

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-24T17:53:54+08:00

There is not enough information for your issue. If you are using RTOS SDK, then you will need to make your stack use comply with max stack usage allowed for tasks. Do let me know of which version of SDK you are using and stack usage. I suspect that is what is happening.
Note that no silent action is ever taken up by SDK. You will get a reboot if there is an issue, always. Unless it is power supply issue, of course.

Statistics: Posted by Guest — Mon Jul 24, 2017 5:53 pm

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-15T02:36:25+08:00

Without enough memory, you can do nothing.

Running an infinite loop and waiting gives developers no clue to what is wrong and how to resolve the issue.
In case of low memory, you can return an error code or message indicating this explicitly.
Silent default behavior will poison everything and should be completely eliminated.

Statistics: Posted by ememberus — Sat Jul 15, 2017 2:36 am

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-06T14:38:32+08:00

Without enough memory, you can do nothing.
Which version of ESP8266 RTOS SDK are you using? Maybe you can try the latest one.

Statistics: Posted by Her Mary — Thu Jul 06, 2017 2:38 pm

Re: (RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-06T12:41:25+08:00

So this is an RTOS SDK issue I guess? Because mbedTLS on nonOS SDK works fine near minimum heap usage limits too, as long as it can allocate memory for operations. And if it cannot, a memory allocation error is generated (and sending fails).
Which version of SDK are you using? Can you attach a log?
I would like to know if this failure occurs during a callback or just before a callback.

For temporarily fixing this, try to check free heap before calling the function that causes failure with low heap space.

Statistics: Posted by Guest — Thu Jul 06, 2017 12:41 pm

(RTOS_SDK) mbedtls_ssl_write low memory issue

2017-07-05T11:45:20+08:00

Hello,

The function mbedtls_ssl_write get stuck forever when there is low heap memory, about 3-4kb of heap. The ssl_write will block forever until you reset the device, it only occurs when the heap memory is low. The interrupts still works while the write is blocking forever. Please investigate.

File -> RTOS_SDK\third_party\mbedtls\library\ssl_tls.c
Function -> int mbedtls_ssl_write( mbedtls_ssl_context *ssl, const unsigned char *buf, size_t len )

Digging a little bit more on the issue, I've added vPortEnterCritical() and vPortExitCritical between the "ssl_write" call, then the code always crashes and prints ShowCritical:1 message on the uart.

Statistics: Posted by gustavomassa — Wed Jul 05, 2017 11:45 am