http server can't handle multipart/form-data -- common for file uploads

[hoisie]

If you try to upload a large file in the http server with any browser, you 
get an error along the lines of:

Failed to parse form data "unknown Content-Type \"multipart/form-data; 
boundary=----WebKitFormBoundaryYcAKGBO01R4EDsKa\""

For large files, most browsers split the upload into smaller pieces. 

You can test this by creating a site such as:

<h3> By File Upload </h3> 
<form enctype="multipart/form-data" action="/file"
method="POST"> 
Choose:
<input name="uploadedfile" type="file" /><br /> <br
/> 
<input type="submit" value="Upload File" /> 
</form>

[hoisie]

Comment 1:

I've attached an example of a multipart file upload. They're easy to generate with 
the poster python package. 
The Content-Type header contains the boundary value:
Content-Length: 59329
Content-Type: multipart/form-data; boundary=a218790fa0c74d9fad6cccb7fafe97cc

Attachments:

1. test-multipart.dat (57344 bytes)

[hoisie]

Comment 2:

I have a working implementation in web.go, want me to send a patch? From the commends 
in http/server.go, somebody might already be working on it?

[rsc]

Comment 3:

I would like to see a discussion of the change
on golang-dev before seeing a patch.
In particular I want to know what the interface is.
Is the file data just going to be kept in memory?
Is there a limit?

Owner changed to [email protected].

Status changed to Accepted.

[mpl]

Comment 4:

How about adding a member "file []byte" to the Request struct and it would be left to
the caller of ParseForm() to first allocate enough mem (through another func) for this
member? that's how you would fix the limit; if not enough is allocated to receive the
file, then ParseForm errors.

[mpl]

Comment 5:

again on the limit issue, I went ahead and did like I proposed above; I added that
upload []byte member (because I only needed to upload one file, but of course it can be
easily adapted for multiple files like in hoisie's web.go), and I allocate a limited
amount (set with a const) of memory for that member before calling ParseForm(). Then in
ParseForm() I return an error if the uploaded file is larger than the limit.

[bradfitz]

Comment 6:

I'd like to propose an interface like this one:
   http://github.com/bradfitz/golang-mime-multipart
It takes the io.Reader from http.Request.Body and gives you an interator of MIME Parts,
each with a Headers map[string]string and the part itself is an io.Reader to read the
bytes of it.
No buffering, no writing to disk.  You can build that stuff on top of it.  This one's
purely an adapter that parses the input as it comes in.  (from http or anywhere that
uses MIME multipart...)

[hoisie]

Comment 7:

Hey Brad, 
Interesting idea. How would this work with ParseForm? Are you suggesting that ParseForm
only work for form-encoded bodies, and then the user would have to use a MultiPartReader
for multipart bodies? Or would this be the mechanism that ParseForm is built upon?

[bradfitz]

Comment 8:

Yes, ParseForm could also be implemented using this, perhaps with options for a
threshold over which to spool to temp disk files.  But at least in my case, I want to
read (and sha1!) contents as they're streamed in to my HTTP server, without doing a
second pass over the data, so I need this lower-level interface directly.

[bradfitz]

Comment 9:

Pending CL for this:
http://golang.org/cl/1681049/show

[hoisie]

Comment 10:

Brad, I like your design better because it allows any program to read multipart data.
When it gets committed I'll abandon my CL.

[bradfitz]

Comment 11:

hoisie, let me know if you have any feedback.  I'm pretty new to Go and I'll admit I
didn't look at web.go (or even find this bug) at the point I started hacking on this.

[mpl]

Comment 12:

cheers for that, it's working fine in my project.

[rsc]

Comment 13:

http://code.google.com/p/go/source/detail?r=4ab63d961945

Status changed to Fixed.