How to Setup Nginx as Mail Proxy on Ubuntu Server

SMTP proxies are specialized mail transfer agents (MTAs) that, similar to other types of proxy servers, pass SMTP sessions through to other MTAs without using the store-and-forward approach of a typical MTA. When an SMTP proxy receives a connection, it initiates another SMTP session to a destination MTA. In this guide we will explain how to setup Nginx as a Mail Proxy on Ubuntu Server 20.04.

Nginx, stylized as NGINX, nginx, or NginX, is a web server that can also be used as a reverse proxy, load balancer, mail proxy, and HTTP cache. The software was created by Igor Sysoev and publicly released in 2004. Nginx is free and open-source software, released under the terms of the 2-clause BSD license. A large fraction of web servers use NGINX, often as a load balancer.

NGINX can proxy IMAP, POP3, and SMTP protocols to one of the upstream mail servers that host mail accounts and thus can be used as a single endpoint for email clients. This may bring in a number of benefits, such as easy scaling the number of mail servers.

Now lets run through how to set up Nginx as a mail proxy on Ubuntu server.
setup Nginx as Mail Proxy

Getting Started with Nginx on Ubuntu Server 20.04

You can simplify your email service and improve its performance with NGINX or NGINX Plus as a proxy for the SMTP, POP3, and IMAP protocols.

setup nginx mail proxy on ubuntu

NGINX can proxy SMTP, POP3, and IMAP protocols to one of the upstream mail servers that host mail accounts and thus can be used as a single endpoint for email clients. This can provide you the following advantages:

  • easy scaling of the number of mail servers
  • choosing a mail server based on different rules
  • distributing the load among mail servers

Nginx Prerequisites

  • IMAP, POP3, and/or SMTP mail servers or an external mail service
  • NGINX Plus (already includes the Mail modules necessary to proxy email traffic) or NGINX Open Source compiled the Mail modules using the –with-mail parameter for email proxy functionality and –with-mail_ssl_module parameter for SSL/TLS support:
				
					$ ./configure --with-mail --with-mail_ssl_module --with-openssl=[DIR]/openssl-1.1.1
				
			

Setup Nginx Mail Proxy

SMTP/IMAP/POP3 Mail Proxy Servers Configuration

Go to the Nginx configuration file and then:

  1. Create a top-level mail context.
  2. Select your mail server name with the server_name directive.
  3. The authentication server authenticates email clients, chooses an upstream server for email processing, and reports errors. Mention the HTTP authentication server with the auth_http directive.
				
					mail {
    server_name mail.example.com;
    auth_http   localhost:9000/cgi-bin/nginxauth.cgi;
    proxy_pass_error_message on;
    #...
}
				
			

Configure each SMTP, IMAP, or POP3 server with the server blocks. For each server, specify the following directives:

  • listen: it specifies the port number corresponding to the specified protocol
  • protocol: it specifies the protocol (you can also leave it because by default it detects from the port specified in the listen directive)
  • smtp_auth, pop3_auth, and imap_auth: it specifies the permitted authentication methods
				
					server {
    listen    25;
    protocol  smtp;
    smtp_auth login plain cram-md5;
} 

server {
    listen    110;
    protocol  pop3;
    pop3_auth plain apop cram-md5;
}

server {
    listen   143;
    protocol imap;
}
				
			

Authentication Setup for Mail Proxy

It is mandatory to have an authentication server for the NGINX mail server proxy. Every client request is first authenticated by an external HTTP authentication server or by an authentication script before SMTP/POP3/IMAP authentication methods. You can create the server by yourself according to the NGINX authentication protocol which is based on the HTTP protocol.

There can be two cases while you try to authenticate the server for the NGINX mail proxy server which are as follows:

  • Authentication Successful: If you get the following response from the server it means that the authentication is successful. The authentication server will choose an upstream server and redirect the request.
				
					HTTP/1.0 200 OK
Auth-Status: OK
Auth-Server:  # the server name or IP address of the upstream server that will used for mail processing
Auth-Port:  # the port of the upstream server
				
			
  • Authentication Failed: If you get the following response or error message from the server it means that the authentication has failed.
				
					HTTP/1.0 200 OK
Auth-Status:  # an error message to be returned to the client, for example "Invalid login or password"
Auth-Wait:  # the number of remaining authentication attempts until the connection is closed
				
			

SSL/TLS Setup for Mail Proxy

SSL stands for Secure Sockets Layer and, in short, it is the standard technology for keeping an internet connection secure and safeguarding any sensitive data that is being sent between two systems, preventing criminals from reading and modifying any information transferred, including potential personal details.

Using SSL/TLS makes sure that data passed between a client and a mail server is secured.

Secure SSL Encryption

To enable SSL/TLS for the mail proxy follow these steps:

  1. In order to configure your NGINX with SSL/TLS support, type the nginx -V command in the command line and then look for the with–mail_ssl_module line in the output.
				
					$ nginx -V
configure arguments: ... with--mail_ssl_module
				
			

2. Put the server certificates that can be obtained from a trusted Certificate Authority (CA) or generated using an SSL library such as OpenSSL and a private key on the server.

3. To enable SSL/TLS for all mail proxy servers, specify the ssl directive and to enable STLS and STARTTLS specify the starttls directive.

				
					ssl on;
				
			

or alternatively;

				
					starttls on;
				
			

4. By using the ssl_certificate directive, specify the path to add the SSL certificates (that are stored in the PEM format), and specify the path to the private key in the ssl_certificate_key directive as depicted below.

				
					mail {
    #...
    ssl_certificate     /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/certs/server.key;
}
				
			

5. It is your choice to use only strong versions and ciphers of SSL/TLS with the ssl_protocols and ssl_ciphers directives, or set your own preferable protocols and ciphers, feel free to choose as per your requirements.

				
					mail {
    #...
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers   HIGH:!aNULL:!MD5;
}
				
			

SSL/TLS Optimization for Mail Proxy

These hints will help you make your NGINX mail proxy faster and more secure.

Set the number of worker processes equal to the number of processors with the worker_processes directive set on the same level as the mail context.

Optionally, you may increase the session lifetime as per your requirement with the ssl_session_timeout directive. By default the session lifetime is set to be 5 minutes only.

Further, to make your NGINX mail proxy faster and more secure, you can follow these steps and specify the following directives:

  1. worker_processes: it sets the number of worker processes equal to the number of processors on the same level as the mail context

2. ssl_session_cache: it enables the shared session cache and disables the built-in session cache

3. ssl_session_timeout: it sets the session lifetime which is 5 minutes by default

				
					worker_processes auto;

mail {
    #...
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 10m;
    #...
}
				
			

Final Thoughts

In this article we discuss how the proxy servers use the HTTP authentication server, and how you can learn to configure HTTP authentication, see here. Furthermore, we discuss the three email proxy servers: SMTP, POP3, and IMAP that are configured with SSL and STARTTLS support. Following is the complete code snippet to setup Nginx as a mail proxy.

				
					worker_processes auto;

mail {
    server_name mail.example.com;
    auth_http   localhost:9000/cgi-bin/nginxauth.cgi;

    proxy_pass_error_message on;

    ssl                 on;
    ssl_certificate     /etc/ssl/certs/server.crt;
    ssl_certificate_key /etc/ssl/certs/server.key;
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 10m;

    server {
        listen     25;
        protocol   smtp;
        smtp_auth  login plain cram-md5;
    }

    server {
        listen    110;
        protocol  pop3;
        pop3_auth plain apop cram-md5;
}

     server {
        listen   143;
        protocol imap;
    }
}
				
			
Avatar for Emad Bin Abid
Emad Bin Abid

I'm a software engineer who has a bright vision and a strong interest in designing and engineering software solutions. I readily understand that in today's agile world the development process has to be rapid, reusable, and scalable; hence it is extremely important to develop solutions that are well-designed and embody a well-thought-of architecture as the baseline. Apart from designing and developing business solutions, I'm a content writer who loves to document technical learnings and experiences so that peers in the same industry can also benefit from them.

0 0 votes
Article Rating
Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x