Force Mail.app to always send UTF-8 messages

Symptom: Every email I send from Apple Mail looks good as far as Apple users are concerned. Any Apple Mail client will display it correctly. However, Windows Outlook users will see a bunch of weird characters (squares, question marks, etc) in my emails.

Reason: Mail should be set to always use Unicode as the default encoding for new emails. This can of course be done manually from the Message » Encoding menu when typing an email and before sending it, but it's a pain to do it manually, and there's no menu option nor preference to set this as default

Solution: Open Terminal and login as root. Then execute the following command to set Unicode as the default encoding:

defaults write com.apple.mail NSPreferredMailCharset "UTF-8"

Restart Mail and try again.

[robg adds: I tested this one, and it doesn't seem to work for me. However, I can't correspond with the Anonymous author, so I'm running it to see if it works for anyone else. Even with this value set, creating a new message and checking the encoding showed it set to "automatic." Can anyone get this to work?]

From our Sponsor...

The macosxhints Poll

What version of OS X are you running as your main OS?

Force Mail.app to always send UTF-8 messages | 20 comments | Create New Account

Click here to return to the 'Force Mail.app to always send UTF-8 messages' hint

The following comments are owned by whomever posted them. This site is not responsible for what they say.

Force Mail.app to always send UTF-8 messages
Authored by: justice on Mon, Aug 13 2007 at 8:14AM PDT

This does work, it's just not obvious, as the change affects what "Automatic" is defined to be. After running the defaults command, I see Content-Type: text/plain; charset=UTF-8; delsp=yes; format=flowed in the message headers, whereas before I had Content-Type: text/plain; charset=ISO-8859-1; format=flowed.

Force Mail.app to always send UTF-8 messages
Authored by: justice on Mon, Aug 13 2007 at 8:16AM PDT

I should also mention that I didn't do this as root, just as my regular user. I don't see why you would need to do it as root unless perhaps you wanted to make the change for all users on your system.

Force Mail.app to always send UTF-8 messages
Authored by: sadyhr on Mon, Aug 13 2007 at 8:34AM PDT

Why should one have to log in as root? I think, that defaults works on the current logged in user, so when you switch to root in the Terminal, you set the behavior for you root user, which you hopefully not use as your daily user ;) /Sebastian

---
/Sebastian

[ Reply to This | # ]

Force Mail.app to always send UTF-8 messages
Authored by: dmmorse on Mon, Aug 13 2007 at 9:04AM PDT

I tried it too, not as root, but as my normal user, and it did not work, not even after quitting Mail before making the change.

Force Mail.app to always send UTF-8 messages
Authored by: justice on Mon, Aug 13 2007 at 9:45AM PDT

Can you send yourself an email and then check the message header to see what encoding is set? As I said in my comment above, I have had success with this, but it only is clear that you were successful if you look at the message headers, not what the encoding submenu shows.

Force Mail.app to always send UTF-8 messages
Authored by: dmmorse on Mon, Aug 13 2007 at 9:12PM PDT

Yes, I sent myself an email to check the message encoding, and it did not work. Also, since I normally have mail default to RTF format, I switched the format to plain text and sent another test email. It still did not work. I have not tried changing the setting as root, but I don't see why that would affect the settings in a single user's account above and beyond changing the setting within the account itself.

Force Mail.app to always send UTF-8 messages
Authored by: jralls on Mon, Aug 13 2007 at 9:46AM PDT

I just did it as myself, and it did work on both my MacBook Pro and my old G5 tower.
Is everyone on 10.4.10?

Force Mail.app to always send UTF-8 messages
Authored by: Pedro Estarque on Mon, Aug 13 2007 at 10:02AM PDT

I have several encoding problems with Mail.app.
For Brazil, the best option is ISO 8859-1 as UTF-8 is not properly understood by yahoo webmail.

Here is how it works, if you use the "defaults write" hack you will send in your chosen format despite the fact that "automatic" is still selected ( you can check you mail headers as noted before). However, if you send your message in rich text format, Mail will encode the plain text part in your chosen format and the html part in ISO 8859-1, which will choke outlook.

I had a tread open discussing about this very issue, but that didn't seem to grab much attention

http://forums.macosxhints.com/showthread.php?t=76364

Force Mail.app to always send UTF-8 messages
Authored by: jiclark on Mon, Aug 13 2007 at 11:59AM PDT

So let me see if I'm correct in my understanding of this so far...

If I have Mail.app set to send in Plain Text, but run the defaults write command above, the messages will be sent encoded as UTF-8. But, as soon as I switch to Rich Text, it will send a two-part message, with each part encoded differently??? That seems just plain weird! Can anyone confirm that? What would the possible point of that be??

I send 99% of my messages as Plain Text, so am I right that it would make sense for me to run the command above anyway, just to be sure my messages appear "clean" for most Windows users?

This is one of those 'hints' that almost needs its own background article to be understandable by all types of users...

Force Mail.app to always send UTF-8 messages
Authored by: Pedro Estarque on Mon, Aug 13 2007 at 5:29PM PDT

I was kind of simplistic in my post. If you set UTF-8 default, it will always send in UTF-8, but that doesn't work for me because of yahoo webmail.

So if you set it to ISO 8859-1 it will work for both plain and html only if you don't add an attachment. If you do, you'll get plain text in UTF-8 and HTML in ISO 8859-1. This is a bug in my opinion, cause it doesn't make any sense. But it is, however, triggered by a non GUI option (meaning a hack, a hack suggested by Apple itself, but hack nevertheless).
http://docs.info.apple.com/article.html?artnum=301986

Force Mail.app to always send UTF-8 messages
Authored by: GaelicWizard on Mon, Aug 13 2007 at 10:43PM PDT

Actually, you're quite wrong about it being a "hack." First, it is a developer-provided option, so its not a hack. Second, it is a developer-supported option, as you yourself showed by linking to the kbase article.

Its just a way of changing what "automatic" means.

JP

---
Pell

Force Mail.app to always send UTF-8 messages
Authored by: Fairly on Mon, Aug 13 2007 at 2:21PM PDT

Off topic but can't resist: as Outlook is the biggest bug farm in the history of the universe I somehow think your choking it is not such a bad thing? Seriously: I don't get why M$ are still peddling that thing, why people are still using it, and why we should have to adjust to them and not to us. Sorry but it does irk me and others. UTF-8 (and not UTF-16) is a good general idea for mail but we'd all have to use it. But I think that will become a standard. Yes Yahoo mail is really bad at that but they're bad at a lot of things. Give them another ten years and they'll catch up. No place for a rant but if you ever have a bad day computing then count your blessings for Apple Mail and mail standards are a giant leap ahead of what everyone else suffers through every day.

Force Mail.app to always send UTF-8 messages
Authored by: Pedro Estarque on Mon, Aug 13 2007 at 5:44PM PDT

I hate Outlook with all my strengths, but Mail.app is one buggy awkward beast.
Just have a quick search for Mail.app in this site and you'll see.
It is amazing how such an old simple thing, such as sending an email, can still be a problem in 2007.
I share your feelings about UTF-8. Why can't we all use it for everything that involves text and be through with it. Standards have a reason to be.
Email needs a reformulation. Let's decide about plain vs html vs text/enriched, and what char encoding to use. And while we are at it, we could have a look at this tiny thing called spam. Web 2.0 ... Email 2.0.

Force Mail.app to always send UTF-8 messages
Authored by: thg on Tue, Aug 14 2007 at 2:58AM PDT

This was all worked out long ago when Tiger was first released. Here is the reference note used extensively in the Apple forums since then:

http://homepage.mac.com/thgewecke/woutlook.html

Force Mail.app to always send UTF-8 messages
Authored by: Pedro Estarque on Tue, Aug 14 2007 at 10:51AM PDT

No it was not. See my post above

Force Mail.app to always send UTF-8 messages
Authored by: thg on Tue, Aug 14 2007 at 3:06PM PDT

Pedro -- Sorry, I don't understand what you mean. What exactly in the woutlook.html note do you not find correct? I of course want to make it as correct as possible regarding the Outlook issue. Send me email if you like (tom at bluesky dot org).

Force Mail.app to always send UTF-8 messages
Authored by: simonpie on Wed, Aug 15 2007 at 8:06AM PDT

Well, it works for me only if I include foreign characters in my email (like ��). Then mail uses UTF-8. Otherwise it uses USASCII.

Force Mail.app to always send UTF-8 messages
Authored by: thg on Wed, Aug 15 2007 at 9:22AM PDT

Yes, in certain circumstances, like when your text only has ascii in it, you need to use Fix C in the note http://homepage.mac.com/thgewecke/woutlook.html, namely put a dingbat or some other character in your text which will force UTF-8.

Force Mail.app to always send UTF-8 messages
Authored by: macgruder on Wed, Aug 15 2007 at 9:43AM PDT

Well, of course if it's ASCII only there's no point in forcing UTF-8, as ASCII and UTF-8 are identical (where they overlap). That was one of the rationales behind the design of UTF-8. UTF-8's first 128 characters have the same binary as ASCII and so even if your computer doesn't understand UTF-8 it'll still understand the most common letters (in English at least).

Force Mail.app to always send UTF-8 messages
Authored by: thg on Wed, Aug 15 2007 at 6:59PM PDT

I think that a text which the composer considers to be ascii (e.g. no accented chars or smart punctuation) may contain non-ascii nbsp's if in Rich Text (=html) in Mail, and these can get misread by Outlook as non-characters, displaying as question marks. In this situation it is desirable to force the message into uniform UTF-8 so as to get the nbsp's read as spaces. I don't think the terminal command will do this, but adding a Unicode dingbat will.

Hint Options

Search

From our Sponsor...

User Functions

What's New:

Hints

Comments last 2 days

Links last 2 weeks

What's New in the Forums?

The Editor's Corner...

Hints by Topic

News from Macworld

The macosxhints Poll