Found a way to upload data VERY fast by wifi

[raduprv]

I am posting this here because I never found the solution online, and I've been looking for days. Before this 'hack', the transfer speed on Windows it was about 6.7 KBps, and on Linux, about 36KBps
Now the speed is very high, something like many mbps, at least 187 KBps, limited by my SD reading time.

So, here is to to send data really fast, on any host:
First, you have to disable the Nagle 'feature', like so:
WiFiClient client; //stuff client.setNoDelay(1);

Then, you need to set the packet size HIGHER than the MTU, which is 1460 bytes. I got good results sending 1760 at a time, but as I said, the bottleneck is the SD card (I am uploading files by ftp). If the size is lower or equal than the MTU, you won't get good results. I think the trick is forcing it to send two packets.

Can someone please try to reproduce my findings? And if it works, maybe this should be documented somewhere (more than here).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[igrr]

Yes, doing it this way certainly works. There is a pull request (#1570) which attempts to speed up WiFiClient transfers so that ESP8266WebServer and other libraries can benefit from this. It's currently blocked by lack of testing though.

[raduprv]

One question would be, why isn't the setNoDelay on by default?
I don't see any good reason for this to be 0.

[baruch]

The nagle algorithm is enabled by default for all systems and all tcp stacks. This is done to protect the network from inadvertant sending of very small packets, with nagle algorithm only the first packet is small and the following ones are batched up thus improving the overall network performance.

It makes sense to keep this behavior as is and allow application developers to override it if they have a good reason to do so.

[raduprv]

Most TCP stacks do not send only one packet and wait for an ACK. They send a few packets at once.
Without this little almost undocumented hack, the transfer speed on windows is really, really bad, at around 6-7KBps

I don't think network congestion is an issue with this MCU, it dosn't have the capacity to saturate a network.

[tuxedo0801]

Vote for keeping current behavior and document the tcpNoDelay option (as it is done also with other languages, like java...).

[raduprv]

Documenting that feature would certainly help a lot. Speaking of which, is there any place with ALL the current esp8266 specific Arduino API?
There's got to be an easier way to find useful functions than reading the source code :)

[tuxedo0801]

I would add it here as a kind of "hint":

https://github.com/esp8266/Arduino/blob/master/doc/libraries.md#wifiesp8266wifi-library

[galed3e3]

I've tried to implement the suggestions (client.setNoDelay(1); and 1760 packet), but am still getting slow uploads...approximately 82k bits/sec, or about 1/10 of the OP's rate. IE 11 is a little bit faster than Firefox 47 (22 sec vs 30 sec to serve a 307kB image from SPIFFS). I'd greatly appreciate some help (or is this the best that can be done from SPIFFS?)

      WiFiClient client = server.available();
      client.setNoDelay(1);
      // stuff...
      client.println("HTTP/1.1 200 OK");
      client.println("Content-Type: image/bmp");
      client.println("Content-Length: 307254");
      client.println();
      webFile = SPIFFS.open("/2016b.BMP", "r");

      if (webFile) {
        const int bufSize = 1760; 
        byte clientBuf[bufSize];
        int clientCount = 0;
        prevTime=millis();

        while (webFile.available())
        {
          clientBuf[clientCount] = webFile.read();
          clientCount++;

          if (clientCount > bufSize-1)
          {          
            client.write((const uint8_t *)clientBuf, bufSize);
            clientCount = 0;
          }
        }
        // final < bufSize byte cleanup packet
        if (clientCount > 0) client.write((const uint8_t *)clientBuf, clientCount);

        // close the file:
        webFile.close();
      } else {
        Serial.println("file !fnd");
      }

[igrr]

Which version are you using? Current git master or one of the releases?

[raduprv]

Try to upload an empty file, like all 0 or something, to see where the bottleneck is.

[igrr]

With the latest git version you should get better performance without using setNoDelay. Just do client.write(webFile);. This will handle buffering internally and send data as fast as it is acked, limited only be network roundtrip time and SPIFFS read time.

[galed3e3]

Thanks for the quick replies. I was using v. 2.2.0... will update to 2.3.0, try client.write(webFile); , and follow-up.