Serving Email Part 5: Autoconfiguration
This is part five of a six part series. In this part we will cover setting up Autoconfiguration for Email Clients.
This series:
- Part 1: The Pieces of the Puzzle
- Part 2: Subdomains and DNS
- Part 3: Encryption and SSL Certificates
- Part 4: The Email Server(s)
- Part 5: Autoconfiguration <- You are here
- Part 6: Debugging Email Servers
This post in the series will cover Autoconfiguration: setting things up so that clients can automatically get the right server settings, bothering the user as little as possible. Preferably they should only need to give the Email Client their email address and password, and it should just work.
There are two major clients that support autoconfiguration: Microsoft Outlook (Microsoft calls it "autodiscovery") and Mozilla Thunderbird. The Apple iPhone does not support autoconfiguration, but it does support importing what they call a "configuration profile". I will be covering all three of these and some other issues surrounding configuration of other clients.
Practically this means that you, as an operator of an Email Server, will want to implement everything you can to minimize user support requests, unless by some lucky chance you're users are limited to some subset of the email clients.
Mozilla Thunderbird
Autoconfiguration
Quick Start
- Right-click and download this XML-file
- Modify it to fit your server settings, deleting sections that don't apply.
- Then, if you can, upload it to your server at this location:
<base-domain>/.well-known/autoconfig/mail/config-v1.1.xml
If your website redirects using ProxyPass / ProxyPassReverse, you can set it up to not redirect that one directory. An example from the Apache settings for this blog:
RewriteEngine On
ProxyPass /.well-known !
ProxyPass / http://localhost:2368/
ProxyPassReverse / http://localhost:2368/
.well-known is a sub-directory that is also used for other files that are not directly connected with website content. For instance for verifying site ownership.
Alternatively, if it is not possible to make the file available at directory .well-known, but you can create a new subdomain, then create the subdomain autoconfig.<base-domain>, and place the configuration file at:
autoconfig.<base-domain>/mail/config-v1.1.xml
Short Explanation
Thunderbird will check the MX record in DNS of the host part of the email address (user@myhost.com). If it finds none, it will use the host part of the email address instead. Then it will try to download the file first from autoconfig.<base-address>/..., then from <base-address>.well-known/... location. I suggest the other way around, because a simple upload is usually easier than creating a new subdomain.
The XML configuration file then has all the information needed to configure the client: the servers, the protocols, the ports and some of the security options (other security options are negotiated at connection).
Original Documentation
- Here is the original documentation from Mozilla
- Here is a direct link to a description of the file format
Microsoft Outlook
Autodiscovery
Quick Start: but probably incomplete
- Right-click and download this XML-file
- Delete the <LoginName> sections.
- Modify it to fit your server settings
- Then upload it to this location:
<base-domain>/autodiscover/autodiscover.xml
Why it is probably incomplete: when the <LoginName> section is missing, Outlook will assume the login name is the username part of the email address without the domain-name (user@mydomain.com). Most modern Email Servers support multiple domains, and consequently require the full email address, with domain, for login.
The Thunderbird autoconfiguration handles this simply: %EMAILADDRESS% substitutes for the full email address. Microsoft Outlook, instead, passes the email address as a parameter, unavailable to non-executing files. If you want to get a complete fix, press on.
Not so Quick Start
In order to retrieve the email address, you need to execute code when Outlook retrieves the autodiscover.xml. If your website supports PHP, you may copy my solution and adjust as needed. If you have something other than PHP, then you will need to write your own code. My solution:
- Download this file
- Rename extension from .txt to .php
- Adjust the fields as needed for your email server.
- Upload it to the autodiscover directory
- Adjust your Apache configuration to retrieve autodiscover.php when autodiscover.xml is requested:
RewriteEngine On RewriteRule autodiscover/autodiscover.xml /autodiscover/autodiscover.php
Short Explanation
When Outlook retrieves autodiscover.xml, it passes the emailaddress and other parameters in the POST data in XML format. POST data may be passed in one of two forms: one is in the address following a "?" - you can simply enter this in the browser - the other is embedded in the POST-exchange - the one that Outlook uses. The PHP I wrote checks both, so that I can test the results using a browser:
Original Documentation
Apple iPhone/iPad/iOS
Configuration Profile
Unless you have an Email Server large enough that Apple includes you in their database, there is no autoconfiguration for you. You have two options: your users enter all the server info by hand, or you give your user a "Configuration Profile", that makes the entries for them.
I've developed a PHP program that will generate a configuration profile and, when a user clicks download on their iPhone, it will start installing.
You can see the program in action here.
You can download the source here. It has the .txt extension so it is not interpreted, you will need to rename it to use it. Note that it is expecting two parameters, emailname and emailaddress. You can modify the program as needed for your website and direct your users to a webpage where they can use it.
Original Documentation
- Here is the original Configuration Profile Reference from Apple (search for "Email Payload")
Other Email Clients
I tested out other clients using an account at dojo-wehrda.de, which has an MX entry to softwareschmiede-herndon.de and a correct Thunderbird config file in .well-known, and a correctly functioning autodiscovery.xml.
So far none have done correct autoconfiguration, despite the standards for DNS SRV entries and the autoconfiguration info presented here. If you have an Email Client that you're fond of, and they don't support autoconfiguration for smaller servers, please inform them about it.
I started out this series by showing screenshots of K-9 Mail and how it should work. Since I use K-9 for multiple email addresses, and some of them automatically configured, I thought they must have some form of autoconfiguration I could use. I was wrong: they have a database of all the major providers and how they should be configured. For smaller Email Servers like mine it will simply throw you into advanced configuration. It is an Open Source project: I intend to talk to them about this and get it fixed or fix it myself.
Next I tried MailDroid, a commercial Android App, with a free ad-based version. It simply said it could not locate the server. And threw me into the incoming/outgoing server dialog. Oh, well.
BlueMail, another commercial Android App with a free version, seemed to work - it didn't complain about anything, and even showed an email from itself congratulating me on my choice. After waiting several minutes for an update that never came, I went to the advanced settings, where I discovered that it had ignored the DNS and the autoconfiguration files and was trying to synchronize via webmail.dojo-wehrda.de. Fail.
Claws-Mail, the Linux-based Email client, fared a little bit better: at first it tried to setup a POP3 account on my local machine, despite my email address. However when I changed the setting to IMAP and pressed [Auto-configure], it correctly found softwareschmiede-herndon.de and its settings. It then proceeded to SMTP, and again had the local machine set as server, without an [Auto-configure] button - to correctly configure outgoing, I would have to enter everything by hand.
The hardest fail was TheBat, a very old Windows-based client - I used it for a number of years with Windows 95. It is a favorite of old programmers. Despite the version being from this year, it failed hard: it did not find softwareschmiede-herndon.de. Clicking the [Testen!] button on the IMAP page with the defaults did not react - no success or error message - it should have told me that the certificate presented was for the wrong server (the certificate is for softwareschmiede-herndon.de, of course). Proceeding to the SMTP page, clicking on [Testen!] caused the dialog to freeze up, so that I ended up using the task manager to kill the process. I used TheBat 32-bit version on Windows 7 Professional, 32-Bit version.
Notes
Regarding DNS and Clients
If you've been following along, you'll know the importance of DNS records regarding Email Server configuration - post #2 was mostly about DNS. Funny thing: a lot of clients cannot access DNS at all, either because of limitations on their platform, or due to being blocked by a firewall. Mozilla Thunderbird skirts the problem by using an MX-lookup server available via HTTP(S).
In Closing
If you've been implementing as you follow along, you should have a functioning server that is easy for users to configure their client software for. If you don't, you'll be happy to know that I've decided to extend this series by yet one more post: Debugging Email Servers.
Thank you for reading my blog post!
Copyright © 2017 William David Herndon. All rights reserved.
This series:
- Part 1: The Pieces of the Puzzle
- Part 2: Subdomains and DNS
- Part 3: Encryption and SSL Certificates
- Part 4: The Email Server(s)
- Part 5: Autoconfiguration <- You are here
- Part 6: Debugging Email Servers