I have been experimenting for a while when I was bumping into my 16M
memory_limit when just sending a 2M attachment.
First I bumped up my memory_limit to 64M to avoid any Errors.
I logged the current memory usage using memory_get_usage at several
checkpoints. Here is a snippet from my logs that shows the
escalating memory usage during the sending process.
Oct 21 14:38:46 /home/f/fungus/public_html/mail/index.php[74]:
E_USER_NOTICE: Memory used: 1729640
Oct 21 14:38:46 /home/f/fungus/public_html/mail/program/steps/mail/
sendmail.inc[2]: E_USER_NOTICE: Memory used: 2593576
Oct 21 14:38:47 /home/f/fungus/public_html/mail/program/steps/mail/
sendmail.inc[231]: E_USER_NOTICE: Memory used: 7514208
Oct 21 14:38:47 /home/f/fungus/public_html/mail/program/include/
rcube_imap.inc[622]: E_USER_NOTICE: Memory used: 10155336
Oct 21 14:38:47 /home/f/fungus/public_html/mail/program/lib/imap.inc
[1886]: E_USER_NOTICE: Memory used: 12796272
Oct 21 14:38:47 /home/f/fungus/public_html/mail/program/lib/imap.inc
[1898]: E_USER_NOTICE: Memory used: 18078104
Oct 21 14:38:48 /home/f/fungus/public_html/mail/program/lib/imap.inc
[1907]: E_USER_NOTICE: Memory used: 18078136
After some searching through the code, and analyzing this same
process with different sized attachments, I have come to a conclusion
that memory usage must be improved.
Currently the memory_limit must be set at 3M plus six times the
maximum message size. 3M for the base application and enough room to
hold the message 6 times. Eek!
Example: for a 2M maximum message size the memory_limit must be 15M. 15M = 3M + 6*2M
Almost 2 whole copies of this message remain in memory in the
"program/steps/mail/sendmail.inc" file. Another copy is stored in
memory for the argument as sent to $IMAP->save_message() in "program/
include/rcube_imap.inc". Another copy for the argument passed to
iil_C_Append() in "program/lib/imap.inc". And 2 more copies are
generated inside that same function.
Solutions:
- destroy the MAIL_MIME object properly in the "program/steps/mail/
sendmail.inc" after it is done with.
2. rework the API to reduce function calls that require the message
body.
3. store the message only once globally, and have all functions read
it instead of passing the message around.
3. consider fixing the Iloha library to manage memory more
efficiently, or using another library.
--lonnie
Does anyone have a clue why this particular statement would create 2
more copies of the message body in memory?
function iil_C_Append(&$conn, $folder, $message){ if (!$folder) return false; $fp = $conn->fp; dbg_memory(); $message = str_replace("\r", "", $message); dbg_memory();
results with a 2.5M message Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1886]: E_USER_NOTICE: Memory used: 12796160 Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1888]: E_USER_NOTICE: Memory used: 18044080
This is the block in the Iloha library that is creating the two extra
copies in memory. Perhaps it is a PHP bug?
--lonnie
1 copy for the function call 1 copy for the return from str_replace
On 10/21/05, Lonnie Olson fungus@aros.net wrote:
Does anyone have a clue why this particular statement would create 2 more copies of the message body in memory?
function iil_C_Append(&$conn, $folder, $message){ if (!$folder) return false; $fp = $conn->fp; dbg_memory(); $message = str_replace("\r", "", $message); dbg_memory();
results with a 2.5M message Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1886]: E_USER_NOTICE: Memory used: 12796160 Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1888]: E_USER_NOTICE: Memory used: 18044080
This is the block in the Iloha library that is creating the two extra copies in memory. Perhaps it is a PHP bug?
--lonnie
-- Christopher A. Watford christopher.watford@gmail.com http://dorm.tunkeymicket.com http://www.theroadtrip2005.com
Hi !!
1 copy for the function call 1 copy for the return from str_replace
if passing $message by reference does not cause any problem it may help to reduce the memory usage.
On 10/21/05, Lonnie Olson fungus@aros.net wrote:
Does anyone have a clue why this particular statement would create 2 more copies of the message body in memory?
function iil_C_Append(&$conn, $folder, $message){ if (!$folder) return false; $fp = $conn->fp; dbg_memory(); $message = str_replace("\r", "", $message); dbg_memory();
results with a 2.5M message Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1886]: E_USER_NOTICE: Memory used: 12796160 Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1888]: E_USER_NOTICE: Memory used: 18044080
This is the block in the Iloha library that is creating the two extra copies in memory. Perhaps it is a PHP bug?
--lonnie
-- Christopher A. Watford christopher.watford@gmail.com http://dorm.tunkeymicket.com http://www.theroadtrip2005.com
On Oct 21, 2005, at 5:32 PM, Christopher A. Watford wrote:
1 copy for the function call 1 copy for the return from str_replace
But the copy for the function call os str_replace() should be
destroyed when the function returns, and the return value should
*replace* the already existing copy in $message.
I agree that there should be some more memory used in between memory
checks, but the *net* change should be zero or less, depending on the
existence of "\r"'s.
--lonnie
On 10/21/05, Lonnie Olson fungus@aros.net wrote:
Does anyone have a clue why this particular statement would create 2 more copies of the message body in memory?
function iil_C_Append(&$conn, $folder, $message){ if (!$folder) return false; $fp = $conn->fp; dbg_memory(); $message = str_replace("\r", "", $message); dbg_memory();
results with a 2.5M message Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1886]: E_USER_NOTICE: Memory used: 12796160 Oct 21 15:44:39 /home/f/fungus/public_html/mail/program/lib/imap.inc [1888]: E_USER_NOTICE: Memory used: 18044080
This is the block in the Iloha library that is creating the two extra copies in memory. Perhaps it is a PHP bug?
On 10/24/05, Lonnie Olson fungus@aros.net wrote:
On Oct 21, 2005, at 5:32 PM, Christopher A. Watford wrote:
1 copy for the function call 1 copy for the return from str_replace
But the copy for the function call os str_replace() should be destroyed when the function returns, and the return value should *replace* the already existing copy in $message.
Just because the str_replace memory is no longer being used does not mean PHP removed it from the current page's working memory set. PHP is lazy to speed up allocation and page runtime.
This is a case for pass-by-reference.
I agree that there should be some more memory used in between memory checks, but the *net* change should be zero or less, depending on the existence of "\r"'s.
--lonnie
-- Christopher A. Watford christopher.watford@gmail.com
On Oct 25, 2005, at 11:03 AM, Christopher A. Watford wrote:
Just because the str_replace memory is no longer being used does not mean PHP removed it from the current page's working memory set. PHP is lazy to speed up allocation and page runtime.
This is a case for pass-by-reference.
Wow, I guess you learn something new every day. I have never *had*
to debug memory usage before. Is there already a bug report for
this error. Perhaps we could work around it by using the pass-by-
reference.
I'll try it out myself and post the results.
--lonnie
-
Christopher A. Watford -
David Saez Padros -
Lonnie Olson