I have noticed that the larger a mailbox gets, the slower roundcube is to respond when trying to read a message. My "Sent" mailbox has 810 messages right now. When trying to read a message from my "Sent" folder it tends to hang a bit longer, but eventually works it's way through...

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards mark@antsclimbtree.com wrote:

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Thanks!

-- Mark Edwards

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards mark@antsclimbtree.com wrote:

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Is your SMTP server on the same box? If so, turn off caching in the config, that helped me out. For record, I have ~1100 messages in my Roundcube folder, and switching from/to this directory seems quick. Oh, and I"m on Freebsd 6.0 with mySQL 4.x, Apache2, Dovecot 0.9

If that doesn't help - what does /var/log/maillog say when this is occuring?

P

http://fak3r.com - you dont have to kick it

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards mark@antsclimbtree.com wrote:

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Thanks!

You are not alone. I have experienced the same issue with Cyrus 2.1.18 and a mailbox of approximately 37,000 messages (the debian-user mailing list). A recent upgrade to Cyrus 2.2.12 didn't seem to change anything. Browsing the folder is fairly snappy but viewing a message is all but impossible. I haven't taken the time to poke through the message retrieval code yet, but I imagine that this can be fixed considering how fast mutt or even Apple Mail can grab a message from the same folder.

Just for the record, I am running Debian 3.1, Mysql 4.1.11, Apache 2.0.54, php 4.3.10, and Cyrus 2.2.12.

On Feb 24, 2006, at 11:24 AM, phil wrote:

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards
mark@antsclimbtree.com wrote:

...

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Is your SMTP server on the same box? If so, turn off caching in
the config, that helped me out. For record, I have ~1100 messages
in my Roundcube folder, and switching from/to this directory seems
quick. Oh, and I"m on Freebsd 6.0 with mySQL 4.x, Apache2, Dovecot
0.9

If that doesn't help - what does /var/log/maillog say when this is
occuring?

Caching is off, and I don't see what it has to do with SMTP. Its not
switching to/from the directory that causes the problem. Its viewing
a message within a directory that has thousands of messages. Viewing
a message in a box with 1000 or so messages is just fine on this
end. I can view a message in a box with 3000 messages, but it takes
a bit longer. If I try to view a message in a box with 7000, it bogs
down and eventually dies with a "Request timed out!" message in the
Roundcube display.

I am not running anything that logs to /var/log/maillog, but my Cyrus
log and my Apache log for the relevant processes are uneventful.
Basically, Roundcube seems to be doing something that eats up memory
and time relative to the size of the mailbox being accessed, and if
the box is sufficiently large, it bogs down my system and times out.

-- Mark Edwards

On Feb 24, 2006, at 11:51 AM, Steve Block wrote:

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards
mark@antsclimbtree.com wrote:

...

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Thanks!

You are not alone. I have experienced the same issue with Cyrus
2.1.18 and a mailbox of approximately 37,000 messages (the debian- user mailing list). A recent upgrade to Cyrus 2.2.12 didn't seem to
change anything. Browsing the folder is fairly snappy but viewing a
message is all but impossible. I haven't taken the time to poke
through the message retrieval code yet, but I imagine that this can
be fixed considering how fast mutt or even Apple Mail can grab a
message from the same folder.

Just for the record, I am running Debian 3.1, Mysql 4.1.11, Apache
2.0.54, php 4.3.10, and Cyrus 2.2.12.

Okay, so based on the responses thus far, I'd say there are two
possibilities here.

1. This is due to some peculiarity between Cyrus and Roundcube, and

people using other IMAP servers don't see this issue. One
interesting thing is that Cyrus's imapd process also explodes in size
during these events, along with the relevant httpd process.

2. The people who see this issue have a wimpy enough server (mine is

quite wimpy - Pentium Pro 180Mhz with 128MB of RAM) that this
situation is enough to bog down the server. Those who don't see it
have a burly enough server to not be affected, even though its still
occurring.

In any case, its fairly obvious that, at least when used with Cyrus,
Roundcube is chewing up resources in a way that it doesn't have to
when viewing a particular message. Memory isn't getting released, or
Roundcube is going through unnecessary steps to get to a particular
message.

If I use Squirrelmail on the same server to access a message in the
same box, and it is just as fast as any other box, or close to it.
No memory explosion in the httpd process, no timeout. Roundcube
shouldn't have to do any more work or suck up any more memory than
Squirrelmail to perform the same action of retrieving a particular
message's data.

I wish I understood AJAX and IMAP well enough to sort this out
myself. Other than this particular problem, Roundcube is fast and
excellent on my machine.

-- Mark Edwards

On Fri, 24 Feb 2006 13:10:10 -0800, Mark Edwards mark@antsclimbtree.com wrote:

On Feb 24, 2006, at 11:24 AM, phil wrote:

...

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards mark@antsclimbtree.com wrote:

...

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Is your SMTP server on the same box? If so, turn off caching in the config, that helped me out. For record, I have ~1100 messages in my Roundcube folder, and switching from/to this directory seems quick. Oh, and I"m on Freebsd 6.0 with mySQL 4.x, Apache2, Dovecot 0.9

If that doesn't help - what does /var/log/maillog say when this is occuring?

Caching is off, and I don't see what it has to do with SMTP. Its not switching to/from the directory that causes the problem.

Yep, I meant IMAP, in I would assume IMAP was hanging up trying to parse through a huge mbox file...but again, if Squirrel doesn't display that, then that's not it.

P

I've seen the same performance problems with SquirrelMail and mailboxes ~5k or greater on a beefy(Dual Opteron, 4GB RAM) v20z server. I've also seen the problem on a webmail client I built, and roudcube.

Here are a couple of questions for you:

Are you possibly accessing your messages via IMAP over NFS(uber bad performance)? Is your disk slow?

• If so can you modify the settings for the drive(e.g. hparm -Tt

/dev/sda to test disk speed )? How much memory is available on the machine you're using? Are you on a slow pipe?

The problem is more than likely a few things, hence the sporadic "it works fine"...."no it doesn't" emails I'm seeing on this subject.

First and foremost.....DISK IO is *HEAVY* with IMAP! Performance will degrade with large amounts of messages, when over NFS, or on a slow drive. Memory...hate to say it...but Apache/PHP likes to chew on memory.
Depending on your configuration(mod_php* vs cgi, etc) you will see more performance issues when parsing thousands of messages over a socket connection to an IMAP server. Might also want to check your php.ini settings and look for memory_limit. Default is 8M, you may want to tweak that...but don't be too liberal..

Mark Edwards wrote:

On Feb 24, 2006, at 11:24 AM, phil wrote:

...

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards
mark@antsclimbtree.com wrote:

...

Roundcube seems to have some difficulty dealing with large mailboxes (large meaning thousands of messages). It can manage indexing one of these, but if I try to actually show a message in a large mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not releasing memory in some way, which becomes a problem as the mailbox size gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Is your SMTP server on the same box? If so, turn off caching in the config, that helped me out. For record, I have ~1100 messages in my Roundcube folder, and switching from/to this directory seems quick.
Oh, and I"m on Freebsd 6.0 with mySQL 4.x, Apache2, Dovecot 0.9

If that doesn't help - what does /var/log/maillog say when this is
occuring?

Caching is off, and I don't see what it has to do with SMTP. Its not
switching to/from the directory that causes the problem. Its viewing
a message within a directory that has thousands of messages. Viewing
a message in a box with 1000 or so messages is just fine on this
end. I can view a message in a box with 3000 messages, but it takes
a bit longer. If I try to view a message in a box with 7000, it bogs
down and eventually dies with a "Request timed out!" message in the
Roundcube display.

I am not running anything that logs to /var/log/maillog, but my Cyrus
log and my Apache log for the relevant processes are uneventful.
Basically, Roundcube seems to be doing something that eats up memory
and time relative to the size of the mailbox being accessed, and if
the box is sufficiently large, it bogs down my system and times out.

-- Mark Edwards

Well, all of that is well and good, but still doesn't explain why the
performance would be so radically worse on the same server and same
mailbox in Roundcube versus Squirrelmail.

I'm not complaining about IMAP performance in general here. IMAP
performance on my server is fine, using every other client. Yes, I'm
in total agreement that IMAP is bound by resources such as disk and
RAM, and will bring a lesser server down if pushed far enough. The
problem here is that Roundcube should be more or less just as good as
any other client regarding this, and it isn't. Its radically worse,
and therefore there is a problem.

No, I'm not using NFS. Let's just assume, for the sake of keeping
this focused, that my IMAP server itself is behaving fine, and
working like a charm with local Squirrelmail and remote clients.

The question that needs to be answered is why Roundcube is not
behaving fine, when everything else is?

On Feb 24, 2006, at 1:46 PM, Benjamin Smith wrote:

I've seen the same performance problems with SquirrelMail and
mailboxes ~5k or greater on a beefy(Dual Opteron, 4GB RAM) v20z
server. I've also seen the problem on a webmail client I built,
and roudcube.

Here are a couple of questions for you:

Are you possibly accessing your messages via IMAP over NFS(uber bad
performance)? Is your disk slow?

• If so can you modify the settings for the drive(e.g. hparm -Tt /

dev/sda to test disk speed )? How much memory is available on the machine you're using? Are you on a slow pipe?

The problem is more than likely a few things, hence the sporadic
"it works fine"...."no it doesn't" emails I'm seeing on this subject.

First and foremost.....DISK IO is *HEAVY* with IMAP! Performance
will degrade with large amounts of messages, when over NFS, or on a
slow drive. Memory...hate to say it...but Apache/PHP likes to chew on memory.
Depending on your configuration(mod_php* vs cgi, etc) you will see
more performance issues when parsing thousands of messages over a
socket connection to an IMAP server. Might also want to check your php.ini settings and look for
memory_limit. Default is 8M, you may want to tweak that...but
don't be too liberal..

Mark Edwards wrote:

...

On Feb 24, 2006, at 11:24 AM, phil wrote:

...

On Fri, 24 Feb 2006 11:09:47 -0800, Mark Edwards
mark@antsclimbtree.com wrote:

...

Roundcube seems to have some difficulty dealing with large
mailboxes (large meaning thousands of messages). It can manage indexing
one of these, but if I try to actually show a message in a large
mailbox, it basically hangs.

When this happens, the browser just sits there waiting for a reply, and I can do a 'top' on the server and watch the apache process
just eat up memory until it finally dies.

It seems that Roundcube is digging through the mail and not
releasing memory in some way, which becomes a problem as the mailbox size
gets large. At least that's my theory.

Does anyone else observe this? This is with Apache 1.3.34, PHP 5.1.2, MySQL 4.1.18 and Cyrus 2.2.12, running on FreeBSD 4.11p14.

Is your SMTP server on the same box? If so, turn off caching in
the config, that helped me out. For record, I have ~1100
messages in my Roundcube folder, and switching from/to this
directory seems quick. Oh, and I"m on Freebsd 6.0 with mySQL
4.x, Apache2, Dovecot 0.9

If that doesn't help - what does /var/log/maillog say when this
is occuring?

Caching is off, and I don't see what it has to do with SMTP. Its
not switching to/from the directory that causes the problem. Its
viewing a message within a directory that has thousands of
messages. Viewing a message in a box with 1000 or so messages is
just fine on this end. I can view a message in a box with 3000
messages, but it takes a bit longer. If I try to view a message
in a box with 7000, it bogs down and eventually dies with a
"Request timed out!" message in the Roundcube display.

I am not running anything that logs to /var/log/maillog, but my
Cyrus log and my Apache log for the relevant processes are
uneventful. Basically, Roundcube seems to be doing something
that eats up memory and time relative to the size of the mailbox
being accessed, and if the box is sufficiently large, it bogs
down my system and times out.

-- Mark Edwards

-- Mark Edwards

I've been wondering for a long time how caching worked (from a high-level), and which of the tables are considered "caching" tables, and which aren't. I assumed that both messages and cache were related to the cache but either that is not the case or the caching flag isn't being properly adhered to.

Can anyone shed any clarity on this?

-Charles

Mark Edwards wrote:

Can you give a very rough outline of the flow involved in listing a particular message? It seems like more is happening than should be happening for this relatively simple operation.

I do see that my "messages" table is filling up with messages, even though I have caching turned off. My "cache" table remains empty though. If caching is off, should Roundcube bother storing any info about any messages at all?

On Feb 24, 2006, at 1:22 PM, Sjon wrote:

...

If the caching-mechanism of Roundcube decreases the performance of the application instead of increasing it; it might be a good idea to improve the caching-mechanism! :) I hope to be able to spend some time on it this weekend; any additional information on the implementation and ideas behind Roundcube current or preferred way of caching would be appreciated

-- Mark Edwards

Hi,

On 25 févr. 06, at 19:11, Thomas Bruederli wrote:

Any suggestions to improve this operation are welcome.

What about setting the prev/next ids in the row when you build the list, and then passing this information back to the show_message command?

I just had a complete look at the code and it seems feasible with some not so trivial modifications. Should I give it a try?

Regards,

-l

On Feb 25, 2006, at 11:26 AM, Leonard Bouchet wrote:

Hi,

On 25 févr. 06, at 19:11, Thomas Bruederli wrote:

...

Any suggestions to improve this operation are welcome.

What about setting the prev/next ids in the row when you build the
list, and then passing this information back to the show_message
command?

I just had a complete look at the code and it seems feasible with
some not so trivial modifications. Should I give it a try?

Just looking at the code in question, it looks like it builds a giant
array with the box's entire index data in it, which obviously will
grow along with the size of the mailbox:

// get previous and next message UID $a_msg_index = $IMAP->message_index(NULL, $_SESSION['sort_col'],
$_SESSION['sort_order']); $MESSAGE['index'] = array_search((string)$_GET['_uid'],
$a_msg_index, TRUE);

if (isset($a_msg_index[$MESSAGE['index']-1])) $javascript .= sprintf("\n%s.set_env('prev_uid', '%s');",
$JS_OBJECT_NAME, $a_msg_index[$MESSAGE['index']-1]); if (isset($a_msg_index[$MESSAGE['index']+1])) $javascript .= sprintf("\n%s.set_env('next_uid', '%s');",
$JS_OBJECT_NAME, $a_msg_index[$MESSAGE['index']+1]);

Shouldn't there be a way to retrieve the UID's in question without
building an array of the entire mailbox? Or maybe I'm missing what
the code is doing.

I'm trying to figure out how the UID's get retrieved when the whole
mailbox is listed, because that's plenty fast. I'm getting lost in
the code though.

-- Mark Edwards

Leonard Bouchet wrote:

Hi,

On 25 févr. 06, at 19:11, Thomas Bruederli wrote:

...

Any suggestions to improve this operation are welcome.

What about setting the prev/next ids in the row when you build the list, and then passing this information back to the show_message command?

I just had a complete look at the code and it seems feasible with some not so trivial modifications. Should I give it a try?

Yes, please! Any suggestions for improvements are welcome!

Regards,

-l

Regards, Thomas

On 26 févr. 06, at 02:13, Mark Edwards wrote:

A little further testing suggests that its the first line that causes the problem:

$a_msg_index = $IMAP->message_index(NULL, $_SESSION['sort_col'], $_SESSION['sort_order']);

Good, that confirms what the problem is.

Having a quick look at message_index in program/include/rcube_imap.inc it appears that it returns an array of the index data for the whole mailbox. Why is this necessary to retrieve the UID for two messages?
Surely there is a better way to do this?

Actually, you're right. I think I just found a much better way to do it using some imap commands. I'll try to make it work and post the appropriate corrections here.

Regards,

-l

Hi,

On 26 févr. 06, at 15:05, Thomas Bruederli wrote:

Mark Edwards wrote:

...

Having a quick look at message_index in program/include/rcube_imap.inc it appears that it returns an array of the index data for the whole mailbox. Why is this necessary to retrieve the UID for two messages? Surely there is a better way to do this?

If you find one, please let me know...

Could you test the attached patch? It basically does 2 things:

1. If sorting is in default mode (date_DESC) or disabled, just use

IMAP's message sequence number to get the next and previous unique IDs. This should be really fast compared to what was done before.

2. If the user sorts the message's list, only do a ' UID SORT' command

instead of 2 ' SORT' and ' FETCH' and get the next and previous IDs from the returned array. I'm not totally convinced this will really improve performance, but may deserve a try. We could also improve things caching the resulting array in the session while we display the list, but I'm not really sure it's a good idea, due to inconsistencies it may add.

It seems to work for me, let me know if it does for you ... and make a backup before applying the patch, you never know :)

Regards,

-l

Note: I also tried to fix some sorting bugs I found in rcube_imap::_list_headers. This could eventually find a way to the main tree also, if Thomas has the time to check it.

On 26 févr. 06, at 23:35, Mark Edwards wrote:

It works great for the most part on my end. Very fast now to view a message in a large box, no different than a small box.

Good.

With the one exception that if the message list is sorted by something other than date_DESC, its pretty much as bad as before -- i.e. total failure on a large box.

I still don't get why it is able to produce the index so quickly no matter what. Why can't the same indexing method be used for both making the main index of a box, and finding the prev/next UID's?

Hum, that's a good question I can't find an answer for the moment. To be sure we correctly identified the problem, could you simply replace the rcube_imap.inc::message_index function with the one below (around line 643)? This one only issues a sort command without checking the cache at all. If the problem persists when you see a new message (with the list sorted by something else than date_DESC), that's effectively strange, because it does a lot less than the _list_headers function, so should be significantly faster, or at least equal.

function message_index($mbox='', $sort_field=NULL, $sort_order=NULL) { if ($sort_field!=NULL) $this->sort_field = $sort_field; if ($sort_order!=NULL) $this->sort_order = strtoupper($sort_order);

 $mailbox = $mbox ? $this->_mod_mailbox($mbox) : $this->mailbox;
 $key = "$mbox:".$this->sort_field.":".$this->sort_order.".msgi";

 // fetch complete message index
 // will only work if sorting is enabled (we normally don't need 

this method with default sorting) if ($this->get_capability('sort') && ($a_index = iil_C_Sort($this->conn, $mailbox, $this->sort_field, '', TRUE))) { if ($this->sort_order == 'DESC') $a_index = array_reverse($a_index);

   $this->cache[$key] = $a_index;
   }

 return $this->cache[$key];
 }

To further test, I would also like you to get the running time of the commands directly sent to the imap server. You could do something like $ telnet imap-server.example.org 143 LOGIN username password SELECT your_huge_mailbox_name SORT (SUBJECT) US-ASCII ALL # how much time does it take to get the answer? UID SORT (SUBJECT) US-ASCII ALL # how much time does it take to get the answer?

Hope this will help finding a better solution,

Regards,

-l

Side note: shouldn't we set $IMAP->skip_deleted to TRUE and always search or sort 'ALL UNDELETED'?

On Feb 26, 2006, at 11:56 PM, Leonard Bouchet wrote:

Hum, that's a good question I can't find an answer for the moment.
To be sure we correctly identified the problem, could you simply
replace the rcube_imap.inc::message_index function with the one
below (around line 643)? This one only issues a sort command
without checking the cache at all. If the problem persists when you
see a new message (with the list sorted by something else than
date_DESC), that's effectively strange, because it does a lot less
than the _list_headers function, so should be significantly faster,
or at least equal.

It was very fast (and of course the prev/next email links didn't
work) so it was the behavior you expected. Significantly faster,
yes, given that otherwise clicking on a message in a large sorted box
is so slow it doesn't complete. With the simplified function, it
took like 1.5 seconds.

To further test, I would also like you to get the running time of
the commands directly sent to the imap server. You could do
something like

$ telnet imap-server.example.org 143 c1 LOGIN username password c2 SELECT your_huge_mailbox_name c3 SORT (SUBJECT) US-ASCII ALL # how much time does it take to get the answer?

c3 OK Completed (12592 msgs in 0.633 secs)

c4 UID SORT (SUBJECT) US-ASCII ALL # how much time does it take to get the answer?

c4 OK Completed (12592 msgs in 0.633 secs)

Its fast.

-- Mark Edwards

Sjon wrote:

I have been trying to optimize RoundCube performance as well; but it is kind of difficult. RoundCube has multiple caches (both php-memory as mysql based) and it isn't always clear where data is coming from.

I think performance can be improved in the _list_headers function. This function requires all headers in a folder to be cached; before this cache will actually be used. This means that if the last of 400 pages has never been visited; RoundCube will never use it's cache of the other 399 pages. I have been trying to improve that; but unfortunately couldn't get it working properly. Viewing a 7000-items folder therefore remains to take 26 seconds to load on my machine.

That's not 100% correct: As long as the cache is not complete, RC has to fetch the *message index* from the server, which should not take too much time. The messages headers are retrieved from the local cache if available. This is done within the method _fetch_headers()

By the way, I just found a performance wasting line which shouldn't be there: you can remove line 564 from rcube_imap.inc ($a_header_index = iil_C_FetchHeaders....) because it just fetches headers for nothing. Must be still there from my experiments when I rewrote the caching functions.

Sorry for that! Thomas

If you would like to check the performance of certain RoundCube actions; insert the following code in your index.php file:

---

[...]

This will insert a timer for both webpages and javascript files.

Regards, Sjon

Thomas Bruederli wrote:

...

Mark Edwards wrote:

...

On Feb 24, 2006, at 1:35 PM, phil wrote: [snip]

Well, in any case, why is doing anything other than saying, "give me this particular message data" and then displaying it?

I guess that the cause of this problem is when RC fetches the message index to show the correct links to next/previous message. This is also done while opening a message and could be related to the mailbox size.

Just to locate the problem, you should remove/comment the lines 68 to 74 in program/steps/mail/show.inc and check if performance is getting better.

I don't have an idea what takes that long in fetching the message index (just doing a SORT command and then fetching the UIDs)

Any suggestions to improve this operation are welcome.

Regards, Thomas

I ended up capturing the network traffic of one of these sessions and found this:

RC sends: "s SORT (DATE) US-ASCII ALL " (note the trailing space) Server responds: "s BAD Invalid Search criteria."

Unlike when I tested manually, the command fails. It turns out to be due to the trailing space(!).

I've attached a patch for this part of the problem.

(Changes $command = 's SORT ('.$field.') US-ASCII ALL '."$add\r\n"; to $command = 's SORT ('.$field.') US-ASCII ALL'.($add?' ':'')."$add\r\n";)

But now I get a blank page (quickly!) and the following in logs/errors:

[21-Apr-2006 00:31:28] PHP Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 35 bytes) in /shares/www/webmail.foo.bar/roundcubetest/program/include/rcube_imap.inc on line 786

Is there any way of making RC consume less memory while doing this?

/Håkan Lindqvist

On Friday 21 April 2006 00:02, Håkan Lindqvist wrote:

I did realize that is a possible workaround. However I find it outrageous for a webmail application to require more than 8 MB of system memory on the server to display a single email. Especially as showing the list of messages doesn't require at all as much memory.

I've not looked at the code in depth with RoundCube yet... however in general, I find your average PHP script kiddie^W^Wdeveloper to be less than frugal with their memory usage in development of php applications of any variety.

Indeed I know of more than one developer that wrote code which loaded the entire contents of a database table (some 80K rows) into an array to do searching on it. This was for an abstraction layer too!

It's absurd.

I'm still hopeful that this memory usage issue can be worked out relatively simply. I just haven't yet been able to find what might be causing it, myself.

I intend to scrutinize the code with a fine tooth comb before I let my (l)users at it and will submit patches accordingly.

I apologise if I've offended anyone, I am being very general.

-- Richard

On fre, 2006-04-28 at 15:01 -0700, Mark Edwards wrote:

Just wanted to chime in here. I applied the latest patch in this
thread to a fresh copy of roundcubemail-cvs-20060413 and all of my
issues went away. Roundcube now speedily displays messages in large
mailboxes (13,000 here). I even have the default 8MB limit in effect
without problems. So, good job as far as I'm concerned.

Ok, good that it helped!

Has this patch been committed yet? Is this issue considered solved?

As far as I know it hasn't been commited yet. I do however suspect that it will be commited as soon as someone gets around to it as I can't possibly think that there could be any bad side effects.

For the record, I also put it in the bug tracking system: http://sourceforge.net/tracker/index.php?func=detail&aid=1475653&gro...

Regards, Håkan Lindqvist