Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Obtaining more than 500 search results in Zimbra Collaboration Server (2016074)

Symptoms

Cross Mailbox Search (from the Admin Console) and the CLI equivalent zmmboxsearch do not yield more than 500 results in Zimbra Collaboration Server (ZCS) 7.x.

For more information, see Zimbra bug 58901.

Purpose

This article provides a workaround to circumvent 500 item limit for search results.

Resolution

The zmmboxsearchx tool was developed to work around this limit. It has functionality similar to zmmboxsearch, but there are some usage differences. See the procedure below for specific instructions.

To work around the 500 item limit for search results:

  1. Download the zmmboxsearchx file, which is attached to Zimbra bug 43265.

  2. Execute all the commands in this procedure as the zimbra user:

    # su - zimbra

  3. Untar the zmmboxsearchx package in a temporary location (for example, in /tmp):

    # cd /tmp
    # tar zxvf zmmboxsearchx-20100625.tgz


    The script is located in the resulting bin/ folder.

  4. To view usage information, run the script without arguments:

    # ./zmmboxsearchx

    zmmboxsearchx: --query is required
    zmmboxsearchx: use of --account, --searchdirectory or <userlistfile> is required
    zmmboxsearchx: try --help option for more information

    Usage:
       zmmboxsearchx [options] [<userlistfile> ...]

         Options: [*] == Required, [+] == Multiple vals OK, (val) == Default
           --query <query_string>   [*]query string
           --dir <directory>        directory to write messages to (no directory)
           --account <acct>         [+]account to include, comma separated values ok
           --exclude <acct>         [+]account to exclude, comma separated values ok
           --exclude-file <file>    [+]file listing accounts to exclude

           --searchdirectory <filt> ldap filter to search for accounts
           --server <hostname>      mailbox server (LC:zimbra_zmprov_default_soap_server)
           --limit <num>            limit the total number of results returned (25)
           --offset <num>           offset in hit list to start at (0)
           --proc <num>             maximum number of child/worker processes to use (4)
           --url <url>              URL of soap service (LC:...)
           --authuser <user>        account for authentication (LC:zimbra_ldap_user)
           --password <password>    password for authuser (LC:zimbra_ldap_password)
           --help                   display a brief help message
           --man                    display the entire man page
           --debug [<num>]          verbose output useful for debugging
           --verbose                increase verbosity (increments --debug)

         Note: LC:<key> means the default is read from localconfig if possible.


    Notes:
    • The options in bold are useful for this procedure in particular; usage will vary according to your specific scenario, however.
    • The default value of 4 for the proc option is sufficient in most scenarios.

  5. Verify the tool's functionality using a subset of the accounts on your system. After confirming that the tool is functioning correctly, try broadening the search to the widest one you need (that is, possibly applying the "*" wildcard).

  6. Usage of the zmmboxsearchx tool differs slightly from the tool which is shipped with the product.

    For this procedure, the command line arguments for zmmboxsearchx are:

    # ./zmmboxsearchx --query "query_string" --account "user_name@domain.com" --dir outputDir

    To search for the phrase "customer list" on all accounts, and with a limit of 999999 results, run the command:

    # ./zmmboxsearchx --query '"customer list"' --account "*" --limit 999999 --dir outputDir

    Note: When searching on multiple words as a whole, surround the phrase with double quotes inside and single quotes outside, as shown in this example.

  7. To import the resulting messages into a destination account so that you can read those messages in a mail/web client, you can use the zmmailbox command.

    For example:

    # zmmailbox -z -m mailsearchresults@example.com am "/Inbox" outputDir/ACCOUNT-ID_MAILITEMID

    480 (/var/tmp/test.txt)


    This adds the email message file ACCOUNT-ID_MAILITEMID to the mailsearchresults@example.com account's Inbox folder (the 480 in the example output above is the message ID assigned to the new message in the database).

    Running this command on each mail item in the folder can be a slow process, as the Java environment is set up each time the zmmailbox command is called. To alleviate this, script this process by creating a zmprov command file which can then be fed to zmmailbox.

    For example, create a text file named cmd.txt which contains lines similar to:

    am "/Inbox" /var/tmp/msg1
    am "/Inbox" /var/tmp/msg2
    am "/Inbox" /var/tmp/msg3


    Then run this command to use the cmd.txt file:

    # zmmailbox -z -m mailsearchresults@example.com < cmd.txt

    The message injection is considerably faster using this method. Once complete, access the mailbox to easily manage the outcome of your search.

Troubleshooting


If you see an error similar to this while executing the script:
SOAPAction: "urn:zimbraAdmin#AuthRequest"

<?xml version="1.0" encoding="UTF-8"?><soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soapenc="http://www.w3.org/2003/05/soap-encoding"  xmlns:xsd="http://www.w3.org/2001/XMLSchema" soap:encodingStyle="http://www.w3.org/2003/05/soap-encoding" xmlns:soap="http://www.w3.org/2003/05/soap-envelope"><soap:Header><context  xmlns="urn:zimbraAdmin" /></soap:Header><soap:Body><AuthRequest xmlns="urn:zimbraAdmin"> <name>zimbra</name><password>*****</password></AuthRequest></soap:Body></soap:Envelope>
SOAP::Transport::HTTP::Client::send_receive: 500 Can't connect to localhost:7071 (certificate verify failed)
Content-Type: text/plain
Client-Date: Wed, 07 Dec 2011 19:49:40 GMT
Client-Warning: Internal response

Can't connect to localhost:7071 (certificate verify failed)

LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /opt/zimbra/zimbramon/lib//LWP/Protocol/http.pm line 51.
zmmboxsearchx: sleep(3): try#2 soap error: 500 Can't connect to localhost:7071 (certificate verify failed) at ./zmmboxsearchx line 851

This indicates that PERL_LWP_SSL_VERIFY_HOSTNAME must be set to zero for the script to run. To resolve this issue, prefix the assignment when calling the script.

For example:

# PERL_LWP_SSL_VERIFY_HOSTNAME=0 zmmboxsearchx --query '"customer list"' --account "*" --limit 999999 --dir outputDir


Additional Information

Run the zmmboxsearchx script on a store node (that is, with the mailboxd service installed). Otherwise, you might be required to install these libraries via CPAN:
  • Parallel::ForkManager
  • SOAP::Lite
To install the libraries (as root), run the commands:

# cpan Parallel::ForkManager
# cpan SOAP::Lite

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 5 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 5 Ratings
Actions
KB: