add ability to specify content-type charset utf-8

[joshunger]

Is it possible to define the Content-Type with a charset of UTF-8 for responses? I don't see it when webpack-dev-server serves content from the content base. This causes encoding issues because the bundle.js is UTF-8.

[sokra]

you can add a meta tag into the HTML.

[joshhunt]

What about for assets that arent HTML?

[ericlau-solid]

I notice that webpack-dev-server's bundle.js is encoded using Windows-1252 while webpack's build output .js files are encoded in UTF-8. This discrepancy is causing some characters to show up one way during development but another on build.

How do you force webpack-dev-server to create bundle.js using UTF-8 encoding? Adding a meta tag in HTML has no effect.

[mikekolganov]

+1 @ericlau-solid

[wildeyes]

@sokra how do you that?

[SpaceK33z]

Does this also happen when outputting a bundle with webpack?

[SpaceK33z]

Is this still happening, and if so, can someone provide more info?

[joshunger]

Did it not repro for you?

On Fri, Sep 9, 2016, 9:34 PM Kees Kluskens [email protected] wrote:

Is this still happening, and if so, can someone provide more info?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#432 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMfl5S-M0cBeURGYalNnde7e-SXEvBLks5qodEBgaJpZM4H1nu2
.

[SpaceK33z]

No, I can't find a simple way to reproduce this. A PR is welcome.

[joshunger]

I'm not on my.comouter for a few days. what content type header do you see on the bundle request?

[SpaceK33z]

The content type header for bundle.js is Content-Type: application/javascript. You want Content-Type: application/javascript; charset=utf-8 I suppose?

I probably don't have this problem because I'm on macOS.

[joshunger]

I'm on Mac too. :) It's only non ASCII (?) special characters that mess you up. Yes. I'm assuming you can look up the content type and dynamically add. That would be great!

[SpaceK33z]

I added ¢Ö in the index.html, and in my entry file. Both ways seem to work, without setting the encoding anywhere. Tested on Chrome, Safari and FF.

[joshunger]

In JavaScript?

On Sat, Sep 10, 2016, 8:40 AM Kees Kluskens [email protected]
wrote:

I added ¢Ö in the index.html, and in my entry file. Both ways seem to
work, without setting the encoding anywhere. Tested on Chrome, Safari and
FF.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#432 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMfl43MskpTo1gWAALkLk88hTDnBkinks5qomzzgaJpZM4H1nu2
.

[SpaceK33z]

Yeah, in examples/modus-inline/app.js, I changed It's working. to It's working.¢Ö. It correctly displayed the special characters without changes.

I noticed that the Javascript file is called with <script src="/bundle.js" type="text/javascript" charset="utf-8"></script>, but even after removing charset="utf-8", the special chars work.

[joshunger]

Ah. Right what is the header on your html file? I believe the browser
defaults the charset to J's files loaded to the charset of the loaded html
file. Send confusing.

On Sat, Sep 10, 2016, 8:54 AM Kees Kluskens [email protected]
wrote:

Yeah, in examples/modus-inline/app.js, I changed It's working. to It's
working.¢Ö. It correctly displayed the special characters without changes.

I noticed that the Javascript file is called with <script src="/bundle.js" type="text/javascript" charset="utf-8"></script>, but
even after removing charset="utf-8", the special chars work.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#432 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMfl7ItDDfsGKNWMroy8xlK9Pylpjiqks5qonAqgaJpZM4H1nu2
.

[SpaceK33z]

Interesting, the header of the html file is Content-Type:text/html; charset=UTF-8. This html file is served by the contentBase option. Do you also use contentBase, or do you use html-webpack-plugin?

[joshunger]

Yeah I don't think we use content base. We have a couple of different
projects that use either html plugin, handlebar template, or nginx.

I believe angular is a good example of a special characters that fails
loading but I can't find the stock overflow post that talked about. They
used a Unicode character for a constant for something.

Apologies for typos.

On Sat, Sep 10, 2016, 8:59 AM Kees Kluskens [email protected]
wrote:

Interesting, the header of the html file is Content-Type:text/html;
charset=UTF-8. This html file is served by the contentBase option. Do you
also use contentBase, or do you use html-webpack-plugin?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#432 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMfl3IhkaqMPnZoMDmzw-73UNEnBzbaks5qonFzgaJpZM4H1nu2
.

[SpaceK33z]

Technically this is an issue in webpack-dev-middleware, that package takes care of making a response. This line is responsible for generating the Content-Type.

I'm not sure though if we can just add charset=UTF-8 to ALL files, without breaking something.

[joshunger]

Yeah how do you look up charset webpack is using?

On Mon, Sep 12, 2016, 10:17 AM Kees Kluskens [email protected]
wrote:

Technically this is an issue in webpack-dev-middleware, that package takes
care of making a response. This line
https://github.com/webpack/webpack-dev-middleware/blob/master/middleware.js#L231
is responsible for generating the Content-Type.

I'm not sure though if we can just add charset=UTF-8 to ALL responses,
without breaking something.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#432 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMfl85bYBz5jHy0UbMO1HPrwgeG1hvWks5qpSadgaJpZM4H1nu2
.

[SpaceK33z]

As far as I can tell, it's only possible to change it with webpack-encoding-plugin. This PR from the webpack repo also assumes utf-8.

[joshunger]

Ah is there a constants file we can just reference? Then we can fix the pr?
Is that what you're thinking?

On Mon, Sep 12, 2016, 10:28 AM Kees Kluskens [email protected]
wrote:

As far as I can tell, it's only possible to change it with
webpack-encoding-plugin
https://github.com/quackes/webpack-encoding-plugin. This PR
https://github.com/webpack/webpack/pull/1797/files from the webpack
repo also assumes utf-8.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#432 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACMflwoeIwQzVCzrA2q08WxoEd8nPeNwks5qpSkzgaJpZM4H1nu2
.

[SpaceK33z]

I'm not sure yet, but after that PR linked above, I'm thinking webpack always assumes utf-8.

[callumacrae]

Is there any workaround or fix to this yet?

After being built with webpack:

Being served with webpack-dev-server:

I can help investigate / fix this if it would be helpful.

I wouldn't class this as a "Nice to Have": it's making it impossible to test my site properly without pushing it to a staging environment.

[SpaceK33z]

@callumacrae, yes it would be nice if you can help with this.

The real issue is in webpack-dev-middleware, to be precise on this line. Probably, it is enough to add ;charset=UTF-8 after the mime lookup.

You could test out if that works. If it works, send a PR. We would still need to find out if it breaks stuff, but we could do a beta release and let some people test it. If it doesn't work, search for other places where Content-Type is set (or should be set) without charset=UTF-8.

[callumacrae]

@SpaceK33z: That fixes the issue for me. I've sent a PR: webpack/webpack-dev-middleware#136