2-way SSL (mutual authentication) using X.509 certificates.

=================
I. Introduction
=================
Two-way SSL authentication, also commonly referred to as SSL mutual authentication, 
is the combination of server and client authentication.  The authentication 
that is occurring is mutual, or two-way, because the server is authenticating 
itself to the client, and the client is authenticating itself to the server.

For a server authenticating itself to the client, the client must trust the 
CA who signed the server's certificate.

For a client authenticating itself to the server, the server must trust the 
CA who signed the client's certificate.


==========
II. Setup
==========
Note: Steps and examples used below are mainly for QA and dev environment.
      We use openssl to generate client certificate and a self-signed 
      CA certificate, then use the CA certificate to sign the client certificate.
      
      We then import the CA certificate into jetty's keystore using keytool.  
      In production environment, zmcertmgr should be used for importing the 
      CA certificate.
      
      The client certificate needs to be imported to the WEB browser.  In examples 
      below, we use Firefox UI as an example. 
      In production environment, the client certificate could be, for example,
      from a CAC card and made available to the browser by some middleware.
    

----------------
0. Preparation
----------------
   Create a temporary directory and cd to the temporary directory, 
   all certificates will be generated in the temporary directory.
   
   mkdir /mycerts
   cd /mycerts
   

----------------
1. Create a Certificate Authority (CA) Certificate
----------------

(A) Create a private key
$ /opt/zimbra/openssl/bin/openssl genrsa -out ca.key 2048
Generating RSA private key, 2048 bit long modulus
....+++
...............................+++
e is 65537 (0x10001)


(B) Create a certificate request
$ /opt/zimbra/openssl/bin/openssl req -new -key ca.key -out ca.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:California
Locality Name (eg, city) []:Palo Alto
Organization Name (eg, company) [Internet Widgits Pty Ltd]:VMWare
Organizational Unit Name (eg, section) []:Zimbra
Common Name (eg, YOUR name) []:MyCA
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:


(C) Create and sign(self-sign) a certificate from the certificate request
$ /opt/zimbra/openssl/bin/openssl x509 -extfile /opt/zimbra/openssl/ssl/openssl.cnf -extensions v3_ca -req -days 365 -in ca.csr -out ca.crt -signkey ca.key
Signature ok
subject=/C=US/ST=California/L=Palo Alto/O=VMWare/OU=Zimbra/CN=MyCA
Getting Private key


(D) Verify that X509v3 extensions are present in the CA cert.
/opt/zimbra/openssl/bin/openssl x509 -in ca.crt -noout -text

Version: 3 (0x2)
...
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                16:C1:9A:E4:31:5B:BA:16:49:CC:99:31:2A:88:D7:6C:B1:B5:45:54
            X509v3 Authority Key Identifier: 
                keyid:16:C1:9A:E4:31:5B:BA:16:49:CC:99:31:2A:88:D7:6C:B1:B5:45:54

            X509v3 Basic Constraints: 
                CA:TRUE
                

----------------
2. Create a Client Certificate
----------------

(A) Create a private key
$ /opt/zimbra/openssl/bin/openssl genrsa -out user1.key 2048
Generating RSA private key, 2048 bit long modulus
........................................................................................................+++
....+++
e is 65537 (0x10001)


(B) Create a certificate request
Note: the most important information is the Email Address.  It must be the email address of the Zimbra user.

$ /opt/zimbra/openssl/bin/openssl req -new -key user1.key -out user1.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:California
Locality Name (eg, city) []:Saratoga
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Example Company
Organizational Unit Name (eg, section) []:Engineering
Common Name (eg, YOUR name) []:user one
Email Address []:user1@phoebe.mbp

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:


(C) Sign the user certificate request using the CA created in 1 and create the user certificate
$ /opt/zimbra/openssl/bin/openssl ca -in user1.csr -cert ca.crt -keyfile ca.key -out user1.crt -policy policy_anything

If you see:
Using configuration from /opt/zimbra/openssl-1.0.0d/ssl/openssl.cnf
I am unable to access the ./demoCA/newcerts directory
./demoCA/newcerts: No such file or directory

This is because the default OpenSSL configuration file uses the ./demoCA/newcerts directory for 
generating new certificates.  OpenSSL also uses certain files to keep track of the last unique 
serial number assigned to a generated certificate and an index of valid and revoked certificates. 

Issue the following commands to setup default contents for these files:
$ mkdir -p ./demoCA/newcerts
$ cd demoCA
$ echo "01" > serial
$ touch index.txt (create an empty index.txt file)
$ cd ..  (so we are back in our temporary directory)

Now issue the command to sign the user certificate again.
$ /opt/zimbra/openssl/bin/openssl ca -in user1.csr -cert ca.crt -keyfile ca.key -out user1.crt -policy policy_anything
Using configuration from /opt/zimbra/openssl-1.0.0d/ssl/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 1 (0x1)
        Validity
            Not Before: Mar 15 05:48:15 2011 GMT
            Not After : Mar 14 05:48:15 2012 GMT
        Subject:
            countryName               = US
            stateOrProvinceName       = California
            localityName              = Saratoga
            organizationName          = Example Company
            organizationalUnitName    = Engineering
            commonName                = user one
            emailAddress              = user1@phoebe.mbp
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Comment: 
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier: 
                57:A8:C2:64:51:7C:C1:3C:5E:01:31:CA:6E:85:A1:11:C2:2D:48:5B
            X509v3 Authority Key Identifier: 
                DirName:/C=US/ST=California/L=Palo Alto/O=VMWare/OU=Zimbra/CN=MyCA
                serial:C2:8A:A1:C3:86:E6:A9:58

Certificate is to be certified until Mar 14 05:48:15 2012 GMT (365 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated


----------------
3. Import the Client Certificate into Web Browsers
----------------

Web browsers like Firefox and IE can't use the certificates in the PEM format that is generated by OpenSSL.
Consequently, we'll need to export the user certificate to file formats that can be imported by web browsers.

(A) Import the client certificate in PKCS#12 format
Firefox and Internet Explorer 6.0 support the PKCS#12 certificate format. 
Use the following command to convert the user certificate to this format.

$ /opt/zimbra/openssl/bin/openssl pkcs12 -export -clcerts -in user1.crt -inkey user1.key -out user1.p12
Enter Export Password:
Verifying - Enter Export Password:
 
Copy the user1.p12 file to a location where you can access it from your web browser via the file system.

(B) To import a certificate in Firefox (Firefox 3.6 Mac):
- Firefox -> preferences
- Click on the Advanced tab
- Under Certificates, select "Ask me every time" for "When a server requests my personal certificate".
- Click on "View Certificates"
- Click on the "Your Certificates" tab
- Click on "import"
- Use the browse button to select the user1.p12 file.  You will be prompted for the password entered 
  in 3. (A).
  
  
----------------
4. Import the CA certificate that signed the client certificate in ZCS's client ssl truststore.
----------------

This can be done by either of the following two ways.

(A) If the client_ssl_truststore localconfig property points to Java's cacerts file:

$ /opt/zimbra/bin/zmcertmgr addcacert <certfile>

    addcacert appends an otherwise untrusted ssl certificate to the cacerts file. 
    This is the preferred way.
    
    This should add the CA to the trust store.
    
    Other related info:
        - Trust store:
        zmlocalconfig -x client_ssl_truststore
        
        - Trust store password:
        zmlocalconfig -s client_ssl_truststore_password

    To verify that the CA is added:
    keytool -list -keystore `zmlocalconfig -x -m nokey client_ssl_truststore` -v -alias {alias-of-the-CA}
    

[========== Begin For dev env only ==========]
On mac, example of the equivalent keytool comamnd that would be issued by 
"zmcertmgr addcacert <certfile>" is:
$ sudo keytool -import -keystore /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/lib/security/cacerts -alias myca -file ca.crt

To verify that the CA is added:
$ keytool -list -keystore /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/lib/security/cacerts -v -alias myca
or 
$ keytool -list -keystore `zmlocalconfig -x -m nokey mailboxd_truststore` -v -alias myca


For dev env on mac, to test providing CA certs from the trust manager, uncomment:
  <Set name="Truststore">/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/lib/security/cacerts</Set>
  <Set name="TrustPassword">changeit</Set>
on the ssl-clientcert SslSelectChannelConnector in /opt/zimbra/jetty/etc/jetty.xml.

[========== End For dev env only ==========]


(B) You can use keytool to import the CA cert to the keystore pointed by
    client_ssl_truststore localconfig property.

$ keytool -import -keystore <client_ssl_truststore> -alias myca -file ca.crt
Enter keystore password:  
Owner: CN=MyCA, OU=Zimbra, O=VMWare, L=Palo Alto, ST=California, C=US
Issuer: CN=MyCA, OU=Zimbra, O=VMWare, L=Palo Alto, ST=California, C=US
Serial number: c28aa1c386e6a958
Valid from: Mon Mar 14 22:17:43 PDT 2011 until: Tue Mar 13 22:17:43 PDT 2012
Certificate fingerprints:
         MD5:  F2:D4:1A:39:0F:1A:72:40:53:03:93:D6:3C:A8:D8:21
         SHA1: 71:A7:96:8D:66:97:10:4D:CF:6F:D8:52:44:BB:5A:E8:38:E5:BA:86
         Signature algorithm name: SHA1withRSA
         Version: 1
Trust this certificate? [no]:  yes
Certificate was added to keystore



To verify that the CA is added:
$ keytool -list -keystore <client_ssl_truststore> -v -alias myca

Other related info:
- Trust store:
  zmlocalconfig -x client_ssl_truststore
        
- Trust store password:
  zmlocalconfig -s client_ssl_truststore_password
        

----------------
5. Configure Zimbra server to request client certificate
----------------
(A) Make sure zimbraMailMode on server is *not* http
$ zmprov gs {server} zimbraMailMode

It is OK as long as zimbraMailMode is not "http".


(B) Configure SSL port for client certificate 

Client certificate authentication happens during SSL handshake and is 
a configuration on the SSL connector(port).  The regular SSL(zimbraMailSSLPort) 
and admin(zimbraAdminPort) port should *not* be configured to request 
client certificate, because SSL mutual authentication will interfere with 
other authentication options on the same port.

SSL mutual authentication must be configured on its own port (e.g. 9443).

To configure the SSL mutual authentication port:

$ zmprov ms {server} zimbraMailSSLClientCertPort {port}


(C) Two modes are supported for client certificate: 
WantClientAuth and NeedClientAuth.
    
Do a "zmprov desc -a zimbraMailSSLClientCertMode" for description 
of this attribute and difference between the two modes.

To configure the client certification mode:
    
$ zmprov ms {server} zimbraMailSSLClientCertMode WantClientAuth
or
$ zmprov ms {server} zimbraMailSSLClientCertMode NeedClientAuth
   
   
(D) Configure Principal map on domain
Domain is resolved by virtual host, set virtual host for a domain:
$ zmprov md {domain} +zimbraVirtualHostname {server}

Configure principal mapping on the domain:
$ zmprov md {domain} zimbraMailSSLClientCertPrincipalMap {mappings}

    Map from a certificate field to a Zimbra account key that can uniquely identify a Zimbra account for client certificate authentication.
    Value is a comma-separated  list of mapping rules, each mapping maps a certificate field to a Zimbra account key.
    Each is attempted in sequence untill a unique account can be resolved.  
    
    e.g. a value can be: 
         SUBJECTALTNAME_OTHERNAME_UPN=zimbraForeignPrincipal,(uid=%{SUBJECT_CN})
    
    value:
        comma-separated mapping-rule
    
    mapping-rule:
        {cert-field-to-zimbra-key-map} | {LDAP-filter}
        
    cert-field-to-zimbra-key-map:     
        {certificate-field}={Zimbra-account-key}
    
    certificate-field:
        SUBJECT_{an RDN attr, e.g. CN}: a RND in DN of Subject
        SUBJECT_DN:                   entire DN of Subject
        SUBJECTALTNAME_OTHERNAME_UPN: UPN(aka Principal Name) in otherName in subjectAltName extension 
        SUBJECTALTNAME_RFC822NAME:    rfc822Name in subjectAltName extension 
    
    Zimbra-account-key:
        name:                   primary name or any of the aliases of an account
        zimbraId:               zimbraId of an account
        zimbraForeignPrincipal: zimbraForeignPrincipal of an account.  
                                The matching value on the zimbraForeignPrincipal must be prefixed with "cert {supported-certificate-filed}:"
                                e.g. cert SUBJECTALTNAME_OTHERNAME_UPN:123456@mydomain
                                
    LDAP-filter: An LDAP filter template with placeholders to be substituted by certificate field values.  
                 (objectClass=zimbraAccount) is internally ANDed with the supplied filter. 
                 e.g. (|(uid=%{SUBJECT_CN})(mail=%{SUBJECTALTNAME_RFC822NAME}))
                 
    Note: it is recommended not to use LDAP-filter rule, as it will trigger an LDAP search for each cert auth request.
          LDAP-filter is disabled by default.  To enable it globally, set zimbraMailSSLClientCertPrincipalMapLdapFilterEnabled 
          on global config to TRUE.  If LDAP-filter is not enabled, all client certificate authentication will fail on domains 
          configured with LDAP-filter.
                            
    Examples of valid mapping:
        SUBJECT_CN=zimbraForeignPrincipal
        SUBJECT_CN=zimbraId
        SUBJECT_CN=name
        SUBJECT_DN=zimbraForeignPrincipal
        SUBJECT_EMAILADDRESS=name
        SUBJECTALTNAME_OTHERNAME_UPN=name
        SUBJECTALTNAME_OTHERNAME_UPN=zimbraForeignPrincipal
        
    Examples of valid value for the attribute:
        - map SUBJECTALTNAME_OTHERNAME_UPN to zimbraForeignPrincipal
        SUBJECTALTNAME_OTHERNAME_UPN=zimbraForeignPrincipal 
        
        - map SUBJECT_EMAILADDRESS to a primary or alias email address of an account, 
          if no match, map SUBJECTALTNAME_OTHERNAME_UPN to zimbraForeignPrincipal
        SUBJECT_EMAILADDRESS=name,SUBJECTALTNAME_OTHERNAME_UPN=zimbraForeignPrincipal   
        
        In the above examples, zimbrasForeignPrincipal for the account must be set to:
        zmprov ma user1@test.com zimbrasForeignPrincipal 'cert SUBJECTALTNAME_OTHERNAME_UPN:123456@mydomain'
        
        - map SUBJECT_CN to zimbraId,
          if no match, do LDAP search to find the entry that has mail equals to SUBJECT_EMAILADDRESS in the certificate.
        SUBJECT_CN=zimbraId,(mail=%SUBJECT_CN)
      
      
To enable LDAP-filter in zimbraMailSSLClientCertPrincipalMap, do          
$ zmprov mcf zimbraMailSSLClientCertPrincipalMapLdapFilterEnabled TRUE

----------------
6. Restart mailbox server
----------------
zmmailboxdctl restart 

   
==============
III. Testing
==============

Note that by default OCSP based certificate revocation checking is mandated for client SSL auth (see OCSP Suppport
section below). This can be disabled by setting zimbraMailSSLClientCertOCSPEnabled provisioning attribute value to
FALSE.

SSL mutual authentication is supported for Zimbra WEB Client(ZWC) and admin console.
You can use either one of the following two ways for testing.

(1) Browse directly to the certauth servlet
    (without the virtual host and login/logout redirect settings)

-----------
Request URL
-----------
ZWC:
    https://{server}:{zimbraMailSSLClientCertPort}/certauth

Admin console:
    https://{server}:{zimbraMailSSLClientCertPort}/certauth/admin
    
    
--------------
Login Behavior
--------------
There are 4 possible scenarios:

1. If a good client certificate is presented, user will be redirected to the 
   requested webapp with a Zimbra auth token.  The webapp will adapt the Zimbra 
   auth token and let user in.
  
2. If client failed to present a client certificate (e.g. there is no matching 
   client certificate in the browser, or user choose to not sent a client certificate)
       - if zimbraMailSSLClientCertMode is WantClientAuth:
             User will be redirected to the requested webapp, without a Zimbra auth token.
             The webapp will show the regular username/password screen so user can login with 
             his username/password if he has a Zimbra password.
        
       - if zimbraMailSSLClientCertMode is NeedClientAuth:
            User will get a SSL handshake error (connection reset by peer).
              
3. If client sends a bad client certificate and server cannot authenticate the client 
   certificate by the CA chain, user will get a SSL handshake error (connection reset by peer) 
   regardless of the zimbraMailSSLClientCertMode.  
  
4. If the client certificate can be authenticated by the CA chain, but the user 
   cannot be authorized by Zimbra (e.g. no such user or zimbraAccountStatus 
   of the user is not active):
       - if zimbraMailSSLClientCertMode is WantClientAuth:
             User will be redirected to the requested webapp, without a Zimbra auth token.
             The webapp will show the regular username/password screen so user can login with 
             his username/password if he has a Zimbra password.
      
       - if zimbraMailSSLClientCertMode is NeedClientAuth:   
             User will get a http 403 FORBIDDEN error.
  

---------------
Logout Behavior
---------------
When the "logout" button is clicked, user will be redirect to the regular entry 
page of the webapp, which is usually the username/password screen.  User can login as 
another user using Zimbra username/password.  


(2) Configure virtual host and login/logout redirect URL on domain, and browse to the 
    default URL for the webapp. 

---------------------
Setup and Request URL
---------------------
ZWC:
    zmprov md {domain} +zimbraVirtualHostname {server}
    zmprov md {domain} zimbraWebClientLoginURL 'https://{server}:{zimbraMailSSLClientCertPort}/certauth'
    zmprov md {domain} zimbraWebClientLogoutURL '../?sso=1'

    browse to: 
    http://{server}:{zimbraMailPort}
    or
    https://{server}:{zimbraMailSSLPort}


Admin console:
    zmprov md {domain} +zimbraVirtualHostname {server}
    zmprov md {domain} zimbraAdminConsoleLoginURL 'https://{server}:{zimbraMailSSLClientCertPort}/certauth/admin'
    zmprov md {domain} zimbraAdminConsoleLogoutURL '../?sso=1'
    
    browse to: 
    https://{server}:{zimbraAdminPort}
    
    Note: This is not yet fully supported for admin console.
    
    Known bug (bug 58163): 
      1. Admin console does not support the "?ignoreLoginURL=1" query.  
         If zimbraMailSSLClientCertMode is WantClientAuth and user choose not 
         to send a client certificate (or there isn't one), we redirect to 
         the admin console entry page.  But if zimbraAdminConsoleLoginURL is 
         set to the certauth URL, it will get into a redirect loop.
         
      2. Admin console does not support "?sso=1" for zimbraAdminConsoleLogoutURL.
         Logout will get into a redirect loop.


Notes on zimbraVirtualHostname:
  You can add virtual host names on domain that are different from the actual Zimbra server name and 
  browse to the virtual hostname.
  e.g. zmprov md {domain} +zimbraVirtualHostname virtual1.company.com +zimbraVirtualHostname virtual2.company.com


--------------
Login Behavior
--------------
First, user will be redirected to the certauth servlet.

then, the 4 possible scenarios descried above for method (1) will apply too.

One point worth noting in the implementation is that whenever we redirect from 
the certauth servlet to the webapp, we always append "?ignoreLoginURL=1" in the 
URL.  This is to prevent possible redirect loop, in cases when the auth token 
is not accepted by the server for any reason.
   
          
---------------
Logout Behavior
---------------
When the "logout" button is clicked, user will be redirect to the page set on the 
zimbraXXXLogoutURL.  The way we set it (i.e.  /?sso=1), the webapp will display a 
page with a "Launch" button.  When "Launch" is clicked, user will be let into the 
webapp again using the client certificate.

            
================================    
SSL mutual authentication flow
================================
- In the handshake the server first proves to the client who it is by signing a
  random challenge sent by the client and returning the corresponding public
  certificate so that the client can check the signature.
  
  In dev/QA environment, the server certificate is a self-signed certificate, if 
  you have not accepted it before in your browser, just accept it (e.g. on Firefox,
  follow the usual "I understand the risks -> add exception, ...").
  If the server certificate is signed by a well known CA, and the CA that signed 
  the server certificate is already in the known Authorities in the browser, 
  you should not be prompted to accept the server certificate.
     
- Then, client certification is requested by the server.  The server sends out 
  descriptions of all the client-issuing-authorities which it has in its key/trust store.
   
- The client now inspects the client certificates which are in the browser and
  attempts to find a match.  A 'match' is a client certificate which was signed by
  one of the authorities which the server says it is already aware of.
  
- At this point, Firefox should display a dialog for you to select a client cert.  
  Select the user1.p12 one imported in step II. 3. (B).
  Firefox now signs the server's random challenge, and returns it and the 
  client's public certificate. 
     
- The server checks that the client certificate was indeed issued by an
  authority it trusts, and checks the signing of the random challenge, dates etc..  
  
- Now the handshake and the "authentication" of user is complete.  
  ZCS will do the "authorization" by looking up the user in ZCS's directory.
  Currently ZCS uses the EMAILADDRESS field of the subject in the client certificate 
  as the only lookup key.  If the value of EMAILADDRESS matches a Zimbra user's 
  primary email address or one of the aliases and the account is in a state good for 
  logging in, the user will be let in.


================
Debugging tips 
================
1. Add -Djavax.net.debug=ssl,handshake,data,trustmanager in localconfig key mailboxd_java_options
   Output will be in zmmailboxd.out.

============
OCSP support 
============

Server uses OCSP protocol to obtain revocation status of X.509 client certificate. All 
the certificates that were issued after 2005-05-16 should have the OCSP Service URL 
automatically included. That is why server checks AuthorityInfoAccess extension of 
certificate to retrieve OCSP responder service url. If aforementioned extension exists, 
OCSP request is made using OCSP responder service url to obtain revocation status of 
certificate in question. Two way certificate based authentication succeeds only in case 
if OCSP returns positive status for submitted certificate. Otherwise if OCSP service is 
down or revoked certificate status is returned, server concludes that client certificate 
is not valid and redirects to password based login page.

=========
OCSP flow 
=========

- As a part of evaluation of client certificate, server retrieves OCSP responder url 
  and sends request containing client certificate along with CA certificate.

- OCSP responder analyzes certificates provided and returns revocation status of certificate

- Based on revocation status, server makes a conclusion in regards to validity of 
  client certificate
   
===============================
Preparation of OCSP environment 
===============================

Preparation of environment for self signed certificates is a multi step process which 
includes setup of local OCSP responder as well as modification of AuthorityInfoAccess 
extension of client certificate which can be somewhat time consuming. Also please note 
that self signed certificate produced as a result of procedure described above doesn't 
contain AuthorityInfoAccess extension necessary for OCSP processing, so the server will
simply skip OCSP check for this particular certificate. Although OCSP feature can be 
tested with any types of certificates (including self signed), luckily i was able to 
find SSL framework called CAcert (http://cacert.org) which significantly simplifies 
preparation of OCSP environment. CAcert hosts OCSP responder (ocsp.cacert.org) listening 
on default OCSP port as well as port 80 for cases where clients are behind the firewall 
and can't reach OCSP on default port, CAcert also offers free SSL certificates pre 
populated with AuthorityInfoAccess extension pointing to http://ocsp.cacert.org. Below go
the steps describing how to setup environment:

- go to http://cacert.org, click on the link called 'Root certificate', find and download 
it in PEM format, save it under mycerts folder as ca.crt (rename existing self signed 
ca.crt before saving).

- import CA certificate into the server's client ssl truststore; refer section 4 (Importing CA certificate).

- create CACert account for testing, go to http://cacert.org and click on 'Join', fill up 
form and provide your zimbra email address, in my case it's eugene@zimbra.com. 

- use credentials created in previous step to login to your account. Go to http://cacert.org
and click on 'Password login'

- create client certificate, click on 'Client Certificates' -> 'New'

- install newly created client certificate in Firefox. After completion of previous step, 
you should see link allowing automatic installation of client certificate. Verify in
Firefox (Firefox->Preferences->View Certificates) that client cert is installed.

- export client cert from Firefox(Firefox->Preferences->View Certificates->double click 
on client certifciate->export->switch to details tab->hit <export> button->save in PEM 
format) and save under mycerts folder as 'clientcert.crt'

- make sure that firewall doesn't block default OCSP port(note that VMware blocks default OCSP port, so usage of public wireless network may help to overcome port problem) and check the validity of client 
certificate using openssl:

openssl ocsp -issuer ca.crt -cert clientcert.crt -url http://ocsp.cacert.org/ -resp_text

If default OCSP port is blocked above command will report connection issue. Otherwise if client 
certificate is valid you should see output similar to below:
 
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: C = AU, ST = NSW, L = Sydney, O = CAcert Inc., OU = Server Administration, CN = ocsp.cacert.org
    Produced At: Apr  6 19:54:56 2012 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 8BA4C9CB172919453EBB8E730991B925F2832265
      Issuer Key Hash: 16B5321BD4C7F3E0E68EF3BDD2B03AEEB23918D1
      Serial Number: 0B90A1
    Cert Status: good
    Revocation Time: Apr  5 06:08:26 2012 GMT
    This Update: Apr  6 19:44:26 2012 GMT
    Next Update: Apr  8 19:54:56 2012 GMT

If certificate is revoked, the output will contain:

    Cert Status: revoked

- make sure that server doesn't run in 'http' mode. use 

zmprov ms <your servername> zimbraMailMode both (or https) to change mailmode

- modify log4j.properties to add tracing info pertinent to OCSP/Two way authentication. 
Add the following line to your existing log4j.properties:

log4j.logger.zimbra.account=DEBUG

Please note that this step is arbitrary and needed only if you need to see trace 
of what is going on during OCSP validation

- restart server for above changes to take effect

- in ZCS (or Octopus) admin console provision new account with email address which was supplied 
to create account on CACert. In my case it is 'eugene@zimbra.com'. This account is necessary for two 
way authentication mechanism to be able to map certificate to ZCS (or Octopus) account.

============
Running test
============

- Using Firefox hit https://<your server name here>:9443/certauth

- Firefox will pop a window stating 'this site has requested to identify yourself with certificate', 
verify certificate's content and hit <ok>.

- At this point, server will use certificate provided by Firefox to perform OCSP validation against 
CACert's OCSP responder

- Since certificate is valid, server automatically authenticates into the account associated
 with client certificate (in my case it is eugene@zimbra.com) without necessity to provide 
username/password credentials

- Now let's revoke client certificate, go to http://cacert.org -> login to previously created
 test account-> cick on 'client certificates'-> hit Revoke/Delete and wait for confirmation 
of certificate revocation

- Verify with openssl that OCSP status of client certificate is 'revoked', use the following 
command to confirm that:

openssl ocsp -issuer ca.crt -cert clientcert.crt -url http://ocsp.cacert.org/ -resp_text

please note that with CACert it may take a couple of minutes(up to 5 minutes) after revocation
of certificate for correct status to show up, so you may need to wait until aforementioned 
command will return proper status

- Once confirmed previous step, restart Firefox as it often caches http responses. Go to
https://<your server name here>:9443/certauth after restart

- Since certificate was revoked, certificate authentication is expected to fail and you 
should be redirected to username/password login page


============
Notes
============

The feature is expected to work on both products ZCS and Octopus. Also you will not see any effects of OCSP 
validation with self signed certificates created to test two way authentication using steps described at the
beginning of this document. These certificates do not contain OCSP url extension and therefore server will
not perform OCSP validation on these certificates.