On Wed, Nov 12, 2008 at 1:23 PM, A.L.E.C alec@alec.pl wrote:

I've made some investigation on messages searching. It was slow and memory expensive. I've found that we can use SORT on servers with that capability (r2046). Also searching with many criterias can be done in one command (r2044). So, in numbers my changes looks as follow:

Searching in forder with 8200 messages, pagesize = 100.

1. Search for "aa" string in subject and from headers, returns 1 message:

Now - Time: 0.8 sec., Mem: 7.2 MB Before - Time: 0.8 sec., Mem: 8.2 MB 2. Search for "aa" string in body, returns 2060 messages: Now - Time: 2.1 sec., Mem: 8.6 MB Before - Time: 5.5 sec., Mem: 21 MB 3. Search for "a" string in subject and from headers, returns 6090 messages: Now - Time: 1.1 sec., Mem: 11 MB Before - Time: 11.8 sec., Mem: 46 MB

So, it looks cool, but I'm not sure that all IMAP servers are supporting searching with SORT, so please do some tests.

p.s. For IMAP servers without SORT capability we need still some work to do. See my comment (search for "TODO") in r2046.

Pretty impressive change. I take it SORT is advertised in the capabilities, right? Do you have a list of imapd's who support this command?

That's something we should add to the wiki - not a list of servers we work with, but a recommendation thing since many people setup from scratch and/or have no issues dropping one for another anyway. ;-)

RE: code, I'd like to sanitize $_SESSION, maybe use get_input_value() (we need another constant for $_SESSION). I couldn't figure out where you initialize them, is that in another diff (didn't see it in r2044 or r2046)?

Great work!

Till _______________________________________________ List info: http://lists.roundcube.net/dev/

On Nov 12, 2008, at 12:34 PM, A.L.E.C wrote:

till wrote:

...

Pretty impressive change. I take it SORT is advertised in the capabilities, right?

Yes. Currenty we're using SORT command for messages list sorting
and now for searching results also.

...

Do you have a list of imapd's who support this command?

I have no such list. SORT is described in RFC 5256. I'm using Dovecot,

In April of this year when use of SORT was first introduced in RC for
sorting, I did some research on a few common IMAP servers to see how
they support SORT. I just reviewed that thread, and did not see any
comments about any other IMAP servers being used besides the three I
researched.

Here is the info in my April post :

Courier - Web site [1] says "IMAP extensions that implement server- side sorting." Cyrus - The Wiki [2] indicates it supports "IMAP SORT and THREAD
Extension" Dovecot - Home page [3] says "Supports multiple commonly used IMAP
extensions, including SORT, THREAD and IDLE."

[1] - http://www.courier-mta.org/imap/features.html [2] - <http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/ SpecificationsAndStandards> [3] - http://dovecot.org/

The Courier page now says "Sorting and searching in non-English
character sets." The Cyrus page doesn't reflect that RFC 5256 was ratified 2008-6-9,
it still refers to the draft. No change relating to SORT on the Dovecot page.

I also researched UW IMAP- http://www.washington.edu/imap/documentation/RELNOTES.html Looks like UW imapd has supported SORT from 2003, and using UTF-8 in
SORT since 2006.

Seeing that, it might be a good idea to test your search function
using characters that are above 8-bit.

Since hMailServer is listed on the Installation page of the Wiki, I
researched that : http://www.hmailserver.com/documentation/v4.2/?page=feature_imap_sort " hMailServer 4.2 and later supports the IMAP sort extension. "

Perhaps the Requirements page - http://trac.roundcube.net/wiki/Howto_Requirements should include mention of SORT support in addtion to requiring "IMAP
4 rev 1"

=====

I use Dovecot here. Are people using other IMAP software, and does that software support
SORT ? There is this list of servers that support IMAP :

<http://en.wikipedia.org/wiki/ Internet_Message_Access_Protocol#Common_Server_implementations> and this list : http://en.wikipedia.org/wiki/List_of_mail_servers#POP.2FIMAP

From that list, it might be worthwhile to research and / or test

MS Exchange CommuniGate Pro IBM Lotus Domino Novell GroupWise Zimbra

On Nov 12, 2008, at 4:08 PM, NetOne - Doichin Dokov wrote:

chasd wrote:

...

Are people using other IMAP software, and does that software support SORT

dBmail also supports SORT. Until some months ago, it was not able to search in the messages' body, but it was fixed. I can confirm RC works just fine with it.

OK. I did some research on dbmail. It didn't support SORT until the 2.2.x
version, and there were issues with SORT after that. I found a Mantis bug report / fix that mentions SquirrelMail, RC and
SORT http://www.dbmail.org/mantis/view.php?id=0000557 That bug was fixed in version 2.2.5.

What version of dbmail did you test with, and did you test the SVN
code mentioned in this thread ?

till wrote:

Sweet, or good to know! :-)

@alec If an imapd doesn't support this (yet), what's gonna happen? ;-)

If IMAP server doesn't support SORT at all, just SEARCH command will be used like before (with big memory consumption, but it can be (partially) fixed in way that I'm proposed in comments). There may be problem if IMAP server does support SORT but doesn't support searching in SORT. I don't know such server.