HTTP File Upload

AgentSmithers
Posts: 155
Joined: Sat Apr 01, 2017 1:21 am
Contact:

HTTP File Upload

Postby AgentSmithers » Wed Jan 23, 2019 12:26 pm

Hi Everyone! I hope everyone had great Holidays!

I am currently working on crafting logic to handle a file upload using the ESP8266 as a Webserver (C NON-OS SDK).
So, While wrapping my head around the protocol it seems pretty straight forward. It sends the HTTP header with a Content-length and a binary blob wrapped between something like this.

Now when I get my first POST
First Packet Init

Code: Select all

POST /upload HTTP/1.1
Host: 192.168.1.71
Connection: keep-alive
Content-Length: 226033
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
Origin: http://192.168.1.51
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary0fb1BQ5qYnyFdhgn
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: http://192.168.1.51/
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9

2nd packet

Code: Select all

0000   a0 20 a6 04 0b be 24 05 0f e3 a5 14 08 00 45 00     ¦..¾$..ã¥...E.
0010   05 dc 10 8b 40 00 80 06 60 fd c0 a8 01 10 c0 a8   .Ü..@...`ýÀ¨..À¨
0020   01 33 c2 b9 00 50 28 bd a1 aa 00 00 19 72 50 10   .3¹.P(½¡ª...rP.
0030   fa f0 e1 f5 00 00 2d 2d 2d 2d 2d 2d 57 65 62 4b   úðáõ..------WebK
0040   69 74 46 6f 72 6d 42 6f 75 6e 64 61 72 79 30 66   itFormBoundary0f
0050   62 31 42 51 35 71 59 6e 79 46 64 68 67 6e 0d 0a   b1BQ5qYnyFdhgn..
0060   43 6f 6e 74 65 6e 74 2d 44 69 73 70 6f 73 69 74   Content-Disposit
0070   69 6f 6e 3a 20 66 6f 72 6d 2d 64 61 74 61 3b 20   ion: form-data;
0080   6e 61 6d 65 3d 22 66 69 6c 65 54 6f 55 70 6c 6f   name="fileToUplo
0090   61 64 22 3b 20 66 69 6c 65 6e 61 6d 65 3d 22 74   ad"; filename="t
00a0   65 73 74 66 6f 74 61 2d 30 78 30 31 30 30 30 2e   estfota-0x01000.
00b0   62 69 6e 22 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79   bin"..Content-Ty
00c0   70 65 3a 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f   pe: application/
00d0   6f 63 74 65 74 2d 73 74 72 65 61 6d 0d 0a 0d 0a   octet-stream....
00e0   ea 04 00 00 04 00 10 40 00 00 00 00 80 05 03 00   ê......@........
00f0   90 83 fe 3f 60 84 fe 3f 08 0e 10 40 6b c0 3f 00   ..þ?`.þ?...@kÀ?.
0100   b8 83 fe 3f 08 84 fe 3f 38 08 00 60 40 10 20 40   ¸.þ?..þ?8..`@. @
0110   98 0f 00 40 50 4c 00 40 94 19 10 40 d0 4c 00 40   ...@PL.@...@ÐL.@
0120   12 c1 f0 09 31 f9 21 fd 01 21 f1 ff 31 f2 ff 01   .Áð.1ù!ý.!ñÿ1òÿ.
...
...
...
...
03b0   70 79 5f 66 72 6f 6d 00 04 08 10 0c 04 08 10 0c   py_from.........
03c0   00 00 20 42 00 00 80 43 00 00 00 00 00 00 20 41   .. B...C...... A
03d0   25 73 20 25 75 0a 00 00 6d 61 69 00 74 69 6d 00   %s %u...mai.tim.
03e0   45 3a 4d 20 25 64 0a 00 25 78 20 61 6c 72 65 61   E:M %d..%x alrea
03f0   64 79 20 66 72 65 65 64 0a 00 00 00 2c 01 00 00   dy freed....,...
0400   18 fe 34 00 ff ff ff ff ff ff 00 00 25 30 32 58   .þ4.ÿÿÿÿÿÿ..%02X
0410   00 00 00 00 25 30 32 78 00 00 00 00 6d 61 63 00   ....%02x....mac.
0420   70 6d 00 00 66 70 6d 00 70 70 00 00 64 65 76 00   pm..fpm.pp..dev.
0430   88 85 fe 3f 50 e8 fe 3f 00 00 00 00 00 00 00 00   ..þ?Pèþ?........
0440   00 00 00 00 00 00 00 a5 0d 0a 2d 2d 2d 2d 2d 2d   .......¥..------
0450   57 65 62 4b 69 74 46 6f 72 6d 42 6f 75 6e 64 61   WebKitFormBounda
0460   72 79 30 66 62 31 42 51 35 71 59 6e 79 46 64 68   ry0fb1BQ5qYnyFdh
0470   67 6e 0d 0a 43 6f 6e 74 65 6e 74 2d 44 69 73 70   gn..Content-Disp
0480   6f 73 69 74 69 6f 6e 3a 20 66 6f 72 6d 2d 64 61   osition: form-da
0490   74 61 3b 20 6e 61 6d 65 3d 22 73 75 62 6d 69 74   ta; name="submit
04a0   22 0d 0a 0d 0a 55 70 6c 6f 61 64 20 49 6d 61 67   "....Upload Imag
04b0   65 0d 0a 2d 2d 2d 2d 2d 2d 57 65 62 4b 69 74 46   e..------WebKitF
04c0   6f 72 6d 42 6f 75 6e 64 61 72 79 30 66 62 31 42   ormBoundary0fb1B
04d0   51 35 71 59 6e 79 46 64 68 67 6e 2d 2d 0d 0a      Q5qYnyFdhgn--..

End of packet, Full size of packet is around 226k bytes




Now, My chip cant easily store Binary blobs in memory due to the constraint of local memory. So the steps I am taking right now is


First Packet Arrived
Parse Content-Length using atoi
Extract Form boundary
Set Global Var to note the remote port and mark it as a File Transfer
The second Packet arrives
seek to CRLFCRLF and remove the bytes seeks from the total content length

REPEAT Until Bytes left is less than MTU(1460)
GET PACKET, Write Data to SPI, Subtract Length from Content-Lengths global var

(When packet is less than MTU, Assumed to be last packet chunk)
REPEAT UNTIL NEXT EIGHT BYTES HAVE \r\n\r\n------ for Mime Border


Anyone have any tips on how to properly parse this data. If I had Vb.Net with 8GB of memory no problem but if you analyze the end of the last packet, I'm wondering if there is a better way to logically go upon this. The Content-length Data removed during the first seek only removes about 60% of the header content from the content the other 40ish percent is at the end but it doesn't appear absolutely predictable as the packet comes in because the true filesize is really unknown. Unless someone can chime in to say maybe that Mime Content is ALWAYS A set amount of bytes for a transfer of one file or something else along those lines that will be very helpful. If you think the check for the 8 Bytes ahead is a performance effective solution then that works for me. I only need to provide for one file in the file upload.
Thank you!!

AgentSmithers
Posts: 155
Joined: Sat Apr 01, 2017 1:21 am
Contact:

Re: HTTP File Upload

Postby AgentSmithers » Wed Feb 06, 2019 1:51 am

I finished this, After getting to the last 3000 bytes I start scanning for the Mime Border at the footer and then I chop it off. Works perfectly for when someone is transferring one file.

Who is online

Users browsing this forum: No registered users and 7 guests