Gmail Tampers with Outgoing Email

Most people who use gmail, Google’s email service, use Google’s web client. They advertise their gmail address (user@gmail.com), and expect it to appear in the From: header of their outgoing email. People use this address to send mail to them, and everything works as it should.

Moving up one level in sophistication, we find people who have their own domains and have configured gmail to construct their outgoing email to use their personal addresses in their From: header. They have probably arranged for incoming mail to their domains to be forwarded to gmail, so they can conduct correspondence using their personal or company email address, and their correspondents will never know they are using gmail. Google makes this straightforward to set up, and everything works as expected.

If you are a serious user of email, of course you don’t want to use a web client, even one as well made as the gmail web client. ¹

The problems begin if you assume that the smtp service that Google offers follows the correct protocol for smtp servers.

The sophisticated user of Mutt and Postfix probably has more than one email address. You may have personal and professional addresses, or several of each. I use a catch-all so I can make up a new email address for each company that needs one, so I can turn them off if they start spamming me, and know when they’ve sold my address.

Real email clients like mutt allow you to edit all the headers of your email, including the From: header, so you can use any address that you want. Since the smtp server that accepts your mail for delivery is not supposed to change these headers, you know that your recipient will see the email with the From: header that you put in, with the email address and identity that you intended to use. This is what the relevant internet standards (see below) require, and this is the behavior that I, at least, have invariably observed with every smtp server that I have ever used.

You also may have your own server to handle incoming mail, but, since smtp-auth can sometimes be a pain to set up, have decided to use the gmail smtp server just for outgoing mail. And once you’ve verified that this is working properly, you may have taken the next step, and set up other users of your incoming mail server to use the gmail smtp server for outgoing mail, all using the same username and password to authenticate.

Imagine the confusion, inconvenience, and possible embarrassment that could be created if the operator of an smtp server decided, unilaterally and without advertisement, to deviate from the published standards and expected behavior by tampering with your email. Imagine if they silently changed the From: header to a different address; one that belonged to someone else, one that was not supposed to be publicly known, or one that is not monitored? Google does exactly this.

All of the scenarios for using more than one email address described above will fail, because Google will rewrite the From: header in all cases to contain the email address that is associated with the authenticated user. And, strangely, it only rewrites the address segment of the header, leaving the personal name intact, and thereby creates headers with personal names and addresses that do not go together.

An example may make this more concrete. Say your name is Joe and your wife’s name is Sally. Sally is trying to send out an email with her iPhone, but whatever she’s using for an smtp server seems to be down. You add your gmail smtp server to her list of secondary outgoing mail servers, which is quick and easy if you’ve done it before, and pure Apple GUI confusion if you haven’t. Now her mail goes out, and you get extra wizard points.

Until she finds out that her email, as received on the other end, had this: From: Sally <joe@gmail.com> and she gets confused notes from her correspondents or replies to her emails go to you.

This happens because you are using your username to authenticate to Google’s servers (Sally doesn’t have her own gmail account); Google knows what email address is established for your account, and has programmed its servers to replace the header you supply with one that contains that address.

Some may be of the opinion that it is wrong to authenticate to the smtp server using one account for more than one person, and that this transgression is the origin of the problems described above. However, this is not an uncommon use pattern; several members of a household, for example, may authenticate to their isp’s smtp server using the same account but maintain separate email identities. Therefore there is no reason to expect an smtp server to behave as if the identity of the account holder has any relationship to the identity of the sender of any particular email.

These are not merely my opinions, but are codified in the various RFPs that define the standards for internet services and protocols.

For example, the document defining the smtp standard states

Efforts to make it more difficult for users to set envelope return path and header “From” fields to point to valid addresses other than their own are largely misguided: they frustrate legitimate applications in which mail is sent by one user on behalf of another, in which error (or normal) replies should be directed to a special address, or in which a single message is sent to multiple recipients on different hosts.

and, even more to the point,

a relay SMTP has no need to inspect or act upon the header section or body of the message data and MUST NOT do so except to add its own “Received:” header field (Section 4.4) and, optionally, to attempt to detect looping in the mail system (see Section 6.3). Of course, this prohibition also applies to any modifications of these header fields or text

So we can see that what Google is doing is exactly what the standard explicitly says that it “MUST NOT” (with upper-case shouting in the original) do. The RFP says that it should not even be looking at the headers, much less fiddling with them.

But we are not using the gmail smtp server as a relay server, exactly. We are submitting mail as an originator to port 587, rather than handing it off to port 25, in most of the cases I’m worrying about here. Does this give them any wiggle room? Not really.

The document defining the standards for message submission is clear that a message submission agent (MSA) should follow the protocol defined for an smtp relay server “with additional restrictions or allowances” that it specifies.

This means that Google would be following the protocols if it rejected an email with a From: address that did not match the address associated with the authenticated user, but is unquestionably violating the protocols by silently rewriting the header, or its address segment.

Of course, Google owns their servers and is under no legal obligation to operate them in accordance with any standards or public protocols that might apply. But to advertise the availability of an smtp server and then to operate it in violation of established protocols and common practice is to behave as a bad internet citizen. Violating users’ expectations may lead, in the scenarios outlined above, to breaches of privacy, embarrassment, and inconvenience. It is possible to imagine situations in which the blindsiding of users by Google’s header rewriting leads to demonstrable damages and consequent civil action. It would be a hard case to make, but stranger things have happened in court.

The Joe and Sally example almost happened to me, but I checked a test message first, and that was how I leaned about the unexpected behavior of the gmail servers. Some quick (yes) Googling revealed that people have been noticing this behavior or variations on it at least since 2009. There have even been bug reports filed, but Google has been silent. It was while I was upgrading a server that I decided to try to use the gmail smtpd for outgoing mail. Needless to say, I’ve retreated from that idea and once again set up my own smtp-auth server using Postfix and Dovecot, which is performing excellently.

The last couple of years have seen a growing enthusiasm for using Platform as a Service (PaaS) providers. It is certainly tempting to let someone else worry about configuring and maintaining infrastructure. But in the cases when you are relying on someone else to provide a fundamental service such as email delivery, you don’t want to encounter surprising behavior. If there are RFPs or other standards documents, you should be able to rely upon your PaaS provider following these as closely as possible, and making it clear where they intend to deviate from the protocol. Google’s behavior in this area is a reminder to be careful where you invest your trust.