Improving Memory Management While Streaming Content

[afnid]

I have a web-server that generates a page that for example, returns 10K of dynamic content.

Initially I was buffering the page, setting the content-length, and then sending it with a single espconn_sent which seemed fine when it was much smaller. With more than one client hitting it, there was an occasional memory allocation error which I initially attributed to fragmentation, but was probably because espconn_sent was doubling the requirements.

I would prefer to buffer up to a packet size and stream the response using http chunked transfer-encoding. This minimizes my memory usage and increases response time by streaming the content as it is generated. Also gets around having to specify the unknown content-length in the response header.

The problem is that espconn_sent is asynchronous and allocates it's own output buffer, so if I buffer 10k of content, espconn_sent allocates another 10k and copies all of the data to be sent but does nothing until control goes back to the cpu. If I send multiple smaller packets, espconn_sent buffers each packet until the return to the main loop.

Potential solutions in order of preference?

1. Something I missed and can't seem to find that does one of the following or better.

2. A synchronous version of espconn_sent so that data does not have to be copied or buffered.

3. An equivalent to the yield() call as provided by esp8266/Arduino to allow the main loop to catch up and flush the buffers.

4. Continue as written hoping I don't exhaust memory.

Dealing with this buffered output makes the complexity go up considerably when you move beyond the trivial case, looking for an elegant solution.

I think right now I am handling the response directly from the recvcb .. if that makes any difference vs running off a post callback.

Thanks..

[kolban]

Howdy,
I had assumed that the buffer I supplied espconn_sent() had to be maintained by my app until I received a notification that the data has been transmitted. What makes you think that a call to espconn_sent() copies the passed in data to its own private buffer? I'm not saying you are mistaken, just curious on what evidence we have that this is what happens as opposed to the alternate that I postulate which is that the buffer must be maintained until transmission completes.

Neil

[afnid]

One of the error responses from espconn_sent is for out of memory which is why I even thought to check.

I looked at the heap size before and after calling espconn_sent and with each write you see the size disappear from the heap. I tried this with different block sizes to see if they would eventually push the data out, no luck.

I purposefully put a spinlock after the espconn_sent and monitored the traffic, nothing got sent before the wdt kicked in.

There is a 1-to-1 relationship between calling espconn_sent and the sentcb, but it is never called until return to the main loop.

So best case right now still appears to write packets of 1460 bytes which makes the maximum memory usage equal to the page size sitting in the espconn write queue plus the size of my output buffer.

Thanks, please let me know if you see anything that contradicts what I observed.

[afnid]

Actually.. a little of both.

I went back and checked because I had setup a test case with only a couple of packets.

The total memory the sdk will allocate in one call is 3326, but when I write blocks of ~1460 I will see two separate large allocations followed by ~40 byte allocations on each call after the first two.

So it looks like that on write they allocate a send buffer up to around 3k, and any writes larger than that they allocate a reference to your buffer to send, so in those cases you have to preserve your buffer until after the sent callbacks finish. If you do one big buffer, there may be two copies of the first ~3k for awhile.

espconn_sent should return the number of bytes copied like all other write related api's so we can use the resources more effectively.


The problem still is that without creating a state machine to return to the main loop, your entire page will have to fit in memory and remain there until notified. Unless all packets are flushed before calling your post callback you can quickly exhaust all resources handling multiple requests.

Managing your own packet send queue would at least make it so you don't need a contiguous block. I am not trying to send anything that won't fit in ram, but I would like to get my memory requirement down to a minimum and still send the optimal number of packets.

Given this, there is not really much advantage with chunked encoding unless you are sending chunks via a post callback, and even in that case it will be a little involved to not fragment packets..

[tve]

You started the thread by saying "I have a web-server that generates a page that for example, returns 10K of dynamic content." You might want to consider whether you can generate the content chunk by chunk using callbacks. Esphttpd supports this paradigm and uses it for large pages. Basically a page handler gets called by the initial recv callback (where you receive the end of the http request) to generate the first 1-2KB of output, when it returns you send that, then you get the sent callback and invoke the handler again saying "hey, generate the next chunk of content". I know, for some handlers this is not so easy.