I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
hi The default setting in php.ini for a script's max runtime is 30 seconds, which doesn't really give that much space..
What I'd find interesting would be some sort of a permanent connection, .. I haven't looked much at the IMAP library roundcube uses, but it seems it should be easily possible to implement a pconnection to the server... so I'd think the idea was doable...
What's wrong with sending out a request every minute or so? I know a couple of sysadmins that would like to keep the connection count down on their server... IMAP is server intensive stuff as it is..
just some thoughts..
-Arnor
On 2/6/07, Brennan Stehling brennan@offwhite.net wrote:
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
Arnór Heiðar wrote:
hi The default setting in php.ini for a script's max runtime is 30 seconds, which doesn't really give that much space..
What I'd find interesting would be some sort of a permanent connection, .. I haven't looked much at the IMAP library roundcube uses, but it seems it should be easily possible to implement a pconnection to the server... so I'd think the idea was doable...
What's wrong with sending out a request every minute or so? I know a couple of sysadmins that would like to keep the connection count down on their server... IMAP is server intensive stuff as it is..
just some thoughts..
-Arnor
On 2/6/07, *Brennan Stehling* <brennan@offwhite.net mailto:brennan@offwhite.net> wrote:
I am not terribly familiar with the lastest in PHP. One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time. Here is the idea. A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity. When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL. And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message. Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that? Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature. -- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
The only way I know to do file-system stuff is to use a cron job that uses the filesystem functions of PHP. You can get the modified file time of a file easy enough.
You'd probably need to be running a cgi to get what you want done.....
~Brett
Yes, my concern is that parsing the IMAP files is intensive especially once a mailbox gets big. As I use RC I notice that when it looks for updates it takes a while. It would be helpful to find a way to speed up the status update.
What I would like it to do is add the header data to the database which will be used most of the time so that it is faster. When it does find new messages in IMAP I would have it return a response right away asking the client to check again soon for the updates. Then once the response is sent it would collect the new messages and place the header data into the database so it is quickly accessible when the next status update comes in.
I use to do a lot of work in Perl and it did not have a way to release the current connection and continue with processing without delaying the client. Can PHP do something like that? Could PHP start an external process to run the update?
One way to check for updates is to check the files the IMAP server uses. On FreeBSD with Cyrus IMAP it is located at /var/spool/imap/users/<username>. That will obviously different for each IMAP server so it will not be an automatic fix for every installation. Is there a faster way to check to just ask the IMAP server if there are new messages? If so the after processing could preload the header data into the database.
Sorry for the random thoughts...
Brennan
On 2/6/07, Arnór Heiðar arnorhs@gmail.com wrote:
hi The default setting in php.ini for a script's max runtime is 30 seconds, which doesn't really give that much space..
What I'd find interesting would be some sort of a permanent connection, . I haven't looked much at the IMAP library roundcube uses, but it seems it should be easily possible to implement a pconnection to the server... so I'd think the idea was doable...
What's wrong with sending out a request every minute or so? I know a couple of sysadmins that would like to keep the connection count down on their server... IMAP is server intensive stuff as it is..
just some thoughts..
-Arnor
On 2/6/07, Brennan Stehling brennan@offwhite.net wrote:
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
Random thoughts = brainstorming :-)
We really can't depend on having access to the mail files. They're usually only accessible to the user the MTA is running on. To access the mail and keep compatibility high we need to stick to imap/pop connections. We also can't depend on having access to updating cron jobs... I'd say..
But I don't really understand what you're getting at because roundcube already puts the messages in a DB... ?
The way it is now it shouldn't really be so heavy on any server. If it's taking you a while to wait for updates it's probably got something to do with your server load .. ?
Arnor
On 2/7/07, Brennan Stehling brennan@offwhite.net wrote:
Yes, my concern is that parsing the IMAP files is intensive especially once a mailbox gets big. As I use RC I notice that when it looks for updates it takes a while. It would be helpful to find a way to speed up the status update.
What I would like it to do is add the header data to the database which will be used most of the time so that it is faster. When it does find new messages in IMAP I would have it return a response right away asking the client to check again soon for the updates. Then once the response is sent it would collect the new messages and place the header data into the database so it is quickly accessible when the next status update comes in.
I use to do a lot of work in Perl and it did not have a way to release the current connection and continue with processing without delaying the client. Can PHP do something like that? Could PHP start an external process to run the update?
One way to check for updates is to check the files the IMAP server uses. On FreeBSD with Cyrus IMAP it is located at /var/spool/imap/users/<username>. That will obviously different for each IMAP server so it will not be an automatic fix for every installation. Is there a faster way to check to just ask the IMAP server if there are new messages? If so the after processing could preload the header data into the database.
Sorry for the random thoughts...
Brennan
On 2/6/07, Arnór Heiðar arnorhs@gmail.com wrote:
hi The default setting in php.ini for a script's max runtime is 30 seconds, which doesn't really give that much space..
What I'd find interesting would be some sort of a permanent connection,
.. I
haven't looked much at the IMAP library roundcube uses, but it seems it should be easily possible to implement a pconnection to the server... so
I'd
think the idea was doable...
What's wrong with sending out a request every minute or so? I know a
couple
of sysadmins that would like to keep the connection count down on their server... IMAP is server intensive stuff as it is..
just some thoughts..
-Arnor
On 2/6/07, Brennan Stehling brennan@offwhite.net wrote:
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
Brennan,
PHP is essentially a CGI scripting language, designed to process HTTP requests and generate responses. Using it to implement some kind of persistent server would be at odds with this design.
If you want a full-time server with durable in-memory state for each client, a JSP/Servlet architecture would be more appropriate. There was a webmail project called JWebMail http://jwebmail.sourceforge.net/ that attempted to do this a few years ago, but I don't think it's been maintained. I started playing with it before I discovered RoundCube, but it was far enough out of date that it would have been a significant effort to pick it up and run with it.
RoundCube already caches messages in MySQL (or another relational database provider) to avoid round trips to the IMAP server. The assumption there is that the MySQL server is located in close proximity with the web server, and therefore has lower communication latency than the (possibly remote) IMAP server.
Finally, attempting to examine the actual mbox or maildir storage from the PHP script would be counterproductive. The raison d'être of the IMAP server is to encapsulate the details of the mail storage and retrieval.
Striving for faster performance is an excellent pursuit, but deviating too far from the fundamental architecture of the application has its own cost...
-Eric
Brennan Stehling wrote:
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
Yes, I am missing something for performance. Maybe I could have selected a different DBD implementation when I built Cyrus. But when it does pull messages from the IMAP server I can see a significant delay.
If it is IMAP and it is unavoidable that is fine, but if there is a different way to query the IMAP server from the PHP side which is more efficient I would like to know what could be done. Has anyone else noticed a bit of a performance delay when pulling new messages or is it snappy for everyone?
Brennan
On 2/6/07, Eric Stadtherr estadtherr@gmail.com wrote:
Brennan,
PHP is essentially a CGI scripting language, designed to process HTTP requests and generate responses. Using it to implement some kind of persistent server would be at odds with this design.
If you want a full-time server with durable in-memory state for each client, a JSP/Servlet architecture would be more appropriate. There was a webmail project called JWebMail http://jwebmail.sourceforge.net/ that attempted to do this a few years ago, but I don't think it's been maintained. I started playing with it before I discovered RoundCube, but it was far enough out of date that it would have been a significant effort to pick it up and run with it.
RoundCube already caches messages in MySQL (or another relational database provider) to avoid round trips to the IMAP server. The assumption there is that the MySQL server is located in close proximity with the web server, and therefore has lower communication latency than the (possibly remote) IMAP server.
Finally, attempting to examine the actual mbox or maildir storage from the PHP script would be counterproductive. The raison d'être of the IMAP server is to encapsulate the details of the mail storage and retrieval.
Striving for faster performance is an excellent pursuit, but deviating too far from the fundamental architecture of the application has its own cost...
-Eric
Brennan Stehling wrote:
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
-- *Eric Stadtherr* estadtherr@gmail.com mailto:estadtherr@gmail.com
I think I understand what is being discussed here, but I would like to suggest a different way of going about it. The basic problem: IMAP latency is the cause of most of the slowness inherent to roundcube. Roundcube already supports caching, but caching always fails on the first lookup (so whenever you have a new email). In my personal situation, I know it takes about 10 seconds to get a new email with roundcube, and I suspect the IMAP server is at fault. However, I don't control the imap server, I just happen to host my own copy of roundcube linked to it.
What if we have a daemon that runs in the background, checking the server for new emails, and then placing them in the database? Basically, have the database cache update as soon as the information is available, instead of as soon as the information is requested. This could significantly cut down on the latency of roundcube, as my database response is very fast This daemon could even be optional, because if roundcube finds the data isn't in the database as expected, it will proceed to request the data as normal. Through this method, you could hide almost all of the IMAP server latency.
Is this worth the effort to implement? Questionable. You could make the argument that roundcube isn't at fault, that you just need to improve your IMAP server. However, the IT department might think the IMAP performance is adequate because most email clients hide it quite well since the download the messages as soon as they are available, and them store them locally for future access. If roundcube implemented this functionality, it would hide the IMAP performance shortfalls just as well.
Rob
On Tue, 6 Feb 2007 23:16:54 -0600, "Brennan Stehling" brennan@offwhite.net wrote:
Yes, I am missing something for performance. Maybe I could have selected a different DBD implementation when I built Cyrus. But when it does pull messages from the IMAP server I can see a significant delay.
If it is IMAP and it is unavoidable that is fine, but if there is a different way to query the IMAP server from the PHP side which is more efficient I would like to know what could be done. Has anyone else noticed a bit of a performance delay when pulling new messages or is it snappy for everyone?
Brennan
On 2/6/07, Eric Stadtherr estadtherr@gmail.com wrote:
Brennan,
PHP is essentially a CGI scripting language, designed to process HTTP requests and generate responses. Using it to implement some kind of persistent server would be at odds with this design.
If you want a full-time server with durable in-memory state for each client, a JSP/Servlet architecture would be more appropriate. There was a webmail project called JWebMail http://jwebmail.sourceforge.net/ that attempted to do this a few years ago, but I don't think it's been maintained. I started playing with it before I discovered RoundCube, but it was far enough out of date that it would have been a significant effort to pick it up and run with it.
RoundCube already caches messages in MySQL (or another relational database provider) to avoid round trips to the IMAP server. The assumption there is that the MySQL server is located in close proximity with the web server, and therefore has lower communication latency than the (possibly remote) IMAP server.
Finally, attempting to examine the actual mbox or maildir storage from the PHP script would be counterproductive. The raison d'être of the
IMAP
server is to encapsulate the details of the mail storage and retrieval.
Striving for faster performance is an excellent pursuit, but deviating too far from the fundamental architecture of the application has its own cost...
-Eric
Brennan Stehling wrote:
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
-- *Eric Stadtherr* estadtherr@gmail.com mailto:estadtherr@gmail.com
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
!DSPAM:1694,45c960e162625986172424!
Hi,
On Tue, 6 Feb 2007 23:16:54 -0600, "Brennan Stehling" brennan@offwhite.net wrote:
Yes, I am missing something for performance. Maybe I could have selected a different DBD implementation when I built Cyrus. But when it does pull messages from the IMAP server I can see a significant delay.
All things aside, my RC install is already a lot faster than Squirrelmail/IMP will ever be. So I am already pleased with the performance. Of course there is room for improvement, e.g. an imap proxy could be the way to go.
Cheers,
Till Klampaeckel e: mailto:klimpong@gmail.com p: +491704018676 l: http://beta.plazes.com/whereis/till
Want to know what's up in Berlin?
Brennan Stehling a écrit :
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
PHP is not a daemon, it's it secret weapon for managing memory leak and bad code. Every time a new PHP born, respond and die.
For file monitoring, there is FAM (File Alteration Monitor) http://fr2.php.net/manual/en/ref.fam.php
But it's out of scope, IMAP is an abstraction, you can't watch file modification, it's IMAP server business, not yours.
IMAP is pull centric, not push. If you wont to do stuff like that, Cyrus (the most complete IMAP implementation) use notifyd, wich send some UDP packet when a mail comes, you can listen UDP port in a php script (not a server one), with a loop and all server stuff wich can put data in MySQL for its brother, php web.
http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html http://pwet.fr/man/linux/commandes/zephyr
M.
Interesting. I suppose I could use a Perl script which sits and listens for notifications and loads data into the MySQL database for me. I was hoping to avoid using a server-side language besides PHP but if PHP is not going to work as a daemon process then Perl seems like the right alternative.
Brennan
On 2/7/07, Mathieu Lecarme mathieu@garambrogne.net wrote:
Brennan Stehling a écrit :
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
PHP is not a daemon, it's it secret weapon for managing memory leak and bad code. Every time a new PHP born, respond and die.
For file monitoring, there is FAM (File Alteration Monitor) http://fr2.php.net/manual/en/ref.fam.php
But it's out of scope, IMAP is an abstraction, you can't watch file modification, it's IMAP server business, not yours.
IMAP is pull centric, not push. If you wont to do stuff like that, Cyrus (the most complete IMAP implementation) use notifyd, wich send some UDP packet when a mail comes, you can listen UDP port in a php script (not a server one), with a loop and all server stuff wich can put data in MySQL for its brother, php web.
http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html http://pwet.fr/man/linux/commandes/zephyr
M.
I guess you can use just about anything to implement that additional performance layer. Whatever floats your boat.
On 2/7/07, Brennan Stehling brennan@offwhite.net wrote:
Interesting. I suppose I could use a Perl script which sits and listens for notifications and loads data into the MySQL database for me. I was hoping to avoid using a server-side language besides PHP but if PHP is not going to work as a daemon process then Perl seems like the right alternative.
Brennan
On 2/7/07, Mathieu Lecarme mathieu@garambrogne.net wrote:
Brennan Stehling a écrit :
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
PHP is not a daemon, it's it secret weapon for managing memory leak and bad code. Every time a new PHP born, respond and die.
For file monitoring, there is FAM (File Alteration Monitor) http://fr2.php.net/manual/en/ref.fam.php
But it's out of scope, IMAP is an abstraction, you can't watch file modification, it's IMAP server business, not yours.
IMAP is pull centric, not push. If you wont to do stuff like that, Cyrus (the most complete IMAP implementation) use notifyd, wich send some UDP packet when a mail comes, you can listen UDP port in a php script (not a server one), with a loop and all server stuff wich can put data in MySQL for its brother, php web.
http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html http://pwet.fr/man/linux/commandes/zephyr
M.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
Yes, it is not exactly a straightforward solution. I would like a more modular mail system which would give us more hooks to handle incoming mail. I know you can create milters, etc, but those are not a trivial thing to build. I once created extensions to the Apache web server with mod_perl which was really quite easy and I was able to make Apache do something the available extensions could not do alone. (SSL secured reverse proxy with LDAP lookup)
There is the James mail server which was created with Java which I believe had plans for plugins. But I cannot really run Java on my FreeBSD 5.x server.
Is there a modular IMAP server which has hooks for languages like Perl, Python or Ruby?
Brennan
On 2/7/07, till klimpong@gmail.com wrote:
I guess you can use just about anything to implement that additional performance layer. Whatever floats your boat.
On 2/7/07, Brennan Stehling brennan@offwhite.net wrote:
Interesting. I suppose I could use a Perl script which sits and listens for notifications and loads data into the MySQL database for me. I was hoping to avoid using a server-side language besides PHP but if PHP is not going to work as a daemon process then Perl seems like the right alternative.
Brennan
On 2/7/07, Mathieu Lecarme mathieu@garambrogne.net wrote:
Brennan Stehling a écrit :
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on the server for the webmail user. Currently the PHP code is only run when there is a request, but it would be helpful to have something running on the server continually which can respond to events in real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for new messages it would update this timestamp. The service running on the server would be away of active sessions and watch their related IMAP accounts for new activity.
When a new message does arrive it can pull the summary data and place it into the MySQL database so that when the webmail client looks for an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also just hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a service? How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of files for updates? Will it just have to poll the directories and files for the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
PHP is not a daemon, it's it secret weapon for managing memory leak and bad code. Every time a new PHP born, respond and die.
For file monitoring, there is FAM (File Alteration Monitor) http://fr2.php.net/manual/en/ref.fam.php
But it's out of scope, IMAP is an abstraction, you can't watch file modification, it's IMAP server business, not yours.
IMAP is pull centric, not push. If you wont to do stuff like that, Cyrus (the most complete IMAP implementation) use notifyd, wich send some UDP packet when a mail comes, you can listen UDP port in a php script (not a server one), with a loop and all server stuff wich can put data in MySQL for its brother, php web.
http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html http://pwet.fr/man/linux/commandes/zephyr
M.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
-- Till Klampaeckel e: mailto:klimpong@gmail.com p: +491704018676 l: http://beta.plazes.com/whereis/till
Want to know what's up in Berlin?
This discussion is going in every direction, so to get some sort of conclusion in it I would think we should begin by defining a policy regarding dependencies and so forth with this software.
I'd think that we'd like as many people to use roundcube as possible. To do this it has to rely on few different software components as possible and the components it relies on should be very common.
Imap server Apache / PHP / mysql
All ideas about things running in the background (eg. cron tabs, special services, even special extensions etc) are all good, but if you make roundcube depend on stuff like that there are not many people that will be able to use/install it.
A few software projects have fallen into the trap of making the software to dependable upon special requirements. Has anybody here tried to install Horde IMP for an example ? There's a special requirements program that has a long list of very special extensions and setup requirements which not many people can both technically set up and/or don't have access to...
Another bad example. Typo3. I actually gave up on this one. Requiring symbolic links, various php extensions, etc (worst part is probably terrible documentation)....
I think we should learn from other projects' mistakes and do things proper..
If anybody is experiencing a lot of delay in receiving the mail etc., there *might* be a bug but most likly the IMAP server of that setup is very very slow. This is not something roundcube could ever be responsible for.
I think as long as the performance is better than Squirrel (which it already is) then the performance is not something we need to worry about.
I'd say keeping the design and architecture _clean_ is a big MUST
Who's with me on that ?!? :-)
best w, Arnor
On 2/7/07, Brennan Stehling brennan@offwhite.net wrote:
Yes, it is not exactly a straightforward solution. I would like a more modular mail system which would give us more hooks to handle incoming mail. I know you can create milters, etc, but those are not a trivial thing to build. I once created extensions to the Apache web server with mod_perl which was really quite easy and I was able to make Apache do something the available extensions could not do alone. (SSL secured reverse proxy with LDAP lookup)
There is the James mail server which was created with Java which I believe had plans for plugins. But I cannot really run Java on my FreeBSD 5.x server.
Is there a modular IMAP server which has hooks for languages like Perl, Python or Ruby?
Brennan
On 2/7/07, till klimpong@gmail.com wrote:
I guess you can use just about anything to implement that additional performance layer. Whatever floats your boat.
On 2/7/07, Brennan Stehling brennan@offwhite.net wrote:
Interesting. I suppose I could use a Perl script which sits and listens for notifications and loads data into the MySQL database for me. I was hoping to avoid using a server-side language besides PHP but if PHP is not going to work as a daemon process then Perl seems like the right alternative.
Brennan
On 2/7/07, Mathieu Lecarme mathieu@garambrogne.net wrote:
Brennan Stehling a écrit :
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on
the
server for the webmail user. Currently the PHP code is only run
when
there is a request, but it would be helpful to have something
running
on the server continually which can respond to events in
real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for
new
messages it would update this timestamp. The service running on
the
server would be away of active sessions and watch their related
IMAP
accounts for new activity.
When a new message does arrive it can pull the summary data and
place
it into the MySQL database so that when the webmail client looks
for
an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also
just
hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a
service?
How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of
files
for updates? Will it just have to poll the directories and files
for
the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
PHP is not a daemon, it's it secret weapon for managing memory leak
and
bad code. Every time a new PHP born, respond and die.
For file monitoring, there is FAM (File Alteration Monitor) http://fr2.php.net/manual/en/ref.fam.php
But it's out of scope, IMAP is an abstraction, you can't watch file modification, it's IMAP server business, not yours.
IMAP is pull centric, not push. If you wont to do stuff like that,
Cyrus
(the most complete IMAP implementation) use notifyd, wich send some
UDP
packet when a mail comes, you can listen UDP port in a php script
(not a
server one), with a loop and all server stuff wich can put data in
MySQL
for its brother, php web.
http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html http://pwet.fr/man/linux/commandes/zephyr
M.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
-- Till Klampaeckel e: mailto:klimpong@gmail.com p: +491704018676 l: http://beta.plazes.com/whereis/till
Want to know what's up in Berlin?
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
I agree.
On 2/7/07, Arnór Heiðar arnorhs@gmail.com wrote:
This discussion is going in every direction, so to get some sort of conclusion in it I would think we should begin by defining a policy regarding dependencies and so forth with this software.
I'd think that we'd like as many people to use roundcube as possible. To do this it has to rely on few different software components as possible and the components it relies on should be very common.
Imap server Apache / PHP / mysql
All ideas about things running in the background (eg. cron tabs, special services, even special extensions etc) are all good, but if you make roundcube depend on stuff like that there are not many people that will be able to use/install it.
A few software projects have fallen into the trap of making the software to dependable upon special requirements. Has anybody here tried to install Horde IMP for an example ? There's a special requirements program that has a long list of very special extensions and setup requirements which not many people can both technically set up and/or don't have access to...
Another bad example. Typo3. I actually gave up on this one. Requiring symbolic links, various php extensions, etc (worst part is probably terrible documentation)....
I think we should learn from other projects' mistakes and do things proper..
If anybody is experiencing a lot of delay in receiving the mail etc., there *might* be a bug but most likly the IMAP server of that setup is very very slow. This is not something roundcube could ever be responsible for.
I think as long as the performance is better than Squirrel (which it already is) then the performance is not something we need to worry about.
I'd say keeping the design and architecture _clean_ is a big MUST
Who's with me on that ?!? :-)
best w, Arnor
On 2/7/07, Brennan Stehling <brennan@offwhite.net > wrote:
Yes, it is not exactly a straightforward solution. I would like a more modular mail system which would give us more hooks to handle incoming mail. I know you can create milters, etc, but those are not a trivial thing to build. I once created extensions to the Apache web server with mod_perl which was really quite easy and I was able to make Apache do something the available extensions could not do alone. (SSL secured reverse proxy with LDAP lookup)
There is the James mail server which was created with Java which I believe had plans for plugins. But I cannot really run Java on my FreeBSD 5.x server.
Is there a modular IMAP server which has hooks for languages like Perl, Python or Ruby?
Brennan
On 2/7/07, till <klimpong@gmail.com > wrote:
I guess you can use just about anything to implement that additional performance layer. Whatever floats your boat.
On 2/7/07, Brennan Stehling < brennan@offwhite.net> wrote:
Interesting. I suppose I could use a Perl script which sits and listens for notifications and loads data into the MySQL database for me. I was hoping to avoid using a server-side language besides PHP but if PHP is not going to work as a daemon process then Perl seems like the right alternative.
Brennan
On 2/7/07, Mathieu Lecarme < mathieu@garambrogne.net> wrote:
Brennan Stehling a écrit :
I am not terribly familiar with the lastest in PHP.
One concept I am considering is a way to keep the state current on
the
server for the webmail user. Currently the PHP code is only run
when
there is a request, but it would be helpful to have something
running
on the server continually which can respond to events in
real-time.
Here is the idea.
A user logs into the system and that updates a timestamp for their date of last activity. Each time the RC webmail client looks for
new
messages it would update this timestamp. The service running on
the
server would be away of active sessions and watch their related
IMAP
accounts for new activity.
When a new message does arrive it can pull the summary data and
place
it into the MySQL database so that when the webmail client looks
for
an update it can just query the database and not MySQL.
And when it does pull the updates from the server it will also
just
hit the database for the summary data and only use the IMAP server when pulling the full message.
Has anyone seen PHP run on the server-side continually as a
service?
How would we go about implementing that?
Also, what facilities would PHP have to monitor a directory of
files
for updates? Will it just have to poll the directories and files
for
the last update date? Since PHP is largely just meant for web applications I doubt it has a file system monitoring feature.
PHP is not a daemon, it's it secret weapon for managing memory leak
and
bad code. Every time a new PHP born, respond and die.
For file monitoring, there is FAM (File Alteration Monitor) http://fr2.php.net/manual/en/ref.fam.php
But it's out of scope, IMAP is an abstraction, you can't watch file modification, it's IMAP server business, not yours.
IMAP is pull centric, not push. If you wont to do stuff like that,
Cyrus
(the most complete IMAP implementation) use notifyd, wich send some
UDP
packet when a mail comes, you can listen UDP port in a php script
(not a
server one), with a loop and all server stuff wich can put data in
MySQL
for its brother, php web.
http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html
http://pwet.fr/man/linux/commandes/zephyr
M.
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
-- Till Klampaeckel e: mailto:klimpong@gmail.com p: +491704018676 l: http://beta.plazes.com/whereis/till
Want to know what's up in Berlin?
-- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
It seems we're saying the same thing... :-)
-Eric
Arnór Heiðar wrote:
This discussion is going in every direction, so to get some sort of conclusion in it I would think we should begin by defining a policy regarding dependencies and so forth with this software.
I'd think that we'd like as many people to use roundcube as possible. To do this it has to rely on few different software components as possible and the components it relies on should be very common.
Imap server Apache / PHP / mysql
All ideas about things running in the background (eg. cron tabs, special services, even special extensions etc) are all good, but if you make roundcube depend on stuff like that there are not many people that will be able to use/install it.
A few software projects have fallen into the trap of making the software to dependable upon special requirements. Has anybody here tried to install Horde IMP for an example ? There's a special requirements program that has a long list of very special extensions and setup requirements which not many people can both technically set up and/or don't have access to...
Another bad example. Typo3. I actually gave up on this one. Requiring symbolic links, various php extensions, etc (worst part is probably terrible documentation)....
I think we should learn from other projects' mistakes and do things proper..
If anybody is experiencing a lot of delay in receiving the mail etc, there *might* be a bug but most likly the IMAP server of that setup is very very slow. This is not something roundcube could ever be responsible for.
I think as long as the performance is better than Squirrel (which it already is) then the performance is not something we need to worry about.
I'd say keeping the design and architecture _clean_ is a big MUST
Who's with me on that ?!? :-)
best w, Arnor
On 2/7/07, *Brennan Stehling* <brennan@offwhite.net mailto:brennan@offwhite.net> wrote:
Yes, it is not exactly a straightforward solution. I would like a more modular mail system which would give us more hooks to handle incoming mail. I know you can create milters, etc, but those are not a trivial thing to build. I once created extensions to the Apache web server with mod_perl which was really quite easy and I was able to make Apache do something the available extensions could not do alone. (SSL secured reverse proxy with LDAP lookup) There is the James mail server which was created with Java which I believe had plans for plugins. But I cannot really run Java on my FreeBSD 5.x server. Is there a modular IMAP server which has hooks for languages like Perl, Python or Ruby? Brennan On 2/7/07, till <klimpong@gmail.com <mailto:klimpong@gmail.com> > wrote: > I guess you can use just about anything to implement that additional > performance layer. > Whatever floats your boat. > > On 2/7/07, Brennan Stehling < brennan@offwhite.net <mailto:brennan@offwhite.net>> wrote: > > Interesting. I suppose I could use a Perl script which sits and > > listens for notifications and loads data into the MySQL database for > > me. I was hoping to avoid using a server-side language besides PHP > > but if PHP is not going to work as a daemon process then Perl seems > > like the right alternative. > > > > Brennan > > > > On 2/7/07, Mathieu Lecarme < mathieu@garambrogne.net <mailto:mathieu@garambrogne.net>> wrote: > > > Brennan Stehling a écrit : > > > > I am not terribly familiar with the lastest in PHP. > > > > > > > > One concept I am considering is a way to keep the state current on the > > > > server for the webmail user. Currently the PHP code is only run when > > > > there is a request, but it would be helpful to have something running > > > > on the server continually which can respond to events in real-time. > > > > > > > > Here is the idea. > > > > > > > > A user logs into the system and that updates a timestamp for their > > > > date of last activity. Each time the RC webmail client looks for new > > > > messages it would update this timestamp. The service running on the > > > > server would be away of active sessions and watch their related IMAP > > > > accounts for new activity. > > > > > > > > When a new message does arrive it can pull the summary data and place > > > > it into the MySQL database so that when the webmail client looks for > > > > an update it can just query the database and not MySQL. > > > > > > > > And when it does pull the updates from the server it will also just > > > > hit the database for the summary data and only use the IMAP server > > > > when pulling the full message. > > > > > > > > Has anyone seen PHP run on the server-side continually as a service? > > > > How would we go about implementing that? > > > > > > > > Also, what facilities would PHP have to monitor a directory of files > > > > for updates? Will it just have to poll the directories and files for > > > > the last update date? Since PHP is largely just meant for web > > > > applications I doubt it has a file system monitoring feature. > > > > > > > PHP is not a daemon, it's it secret weapon for managing memory leak and > > > bad code. Every time a new PHP born, respond and die. > > > > > > For file monitoring, there is FAM (File Alteration Monitor) > > > http://fr2.php.net/manual/en/ref.fam.php > > > > > > But it's out of scope, IMAP is an abstraction, you can't watch file > > > modification, it's IMAP server business, not yours. > > > > > > IMAP is pull centric, not push. If you wont to do stuff like that, Cyrus > > > (the most complete IMAP implementation) use notifyd, wich send some UDP > > > packet when a mail comes, you can listen UDP port in a php script (not a > > > server one), with a loop and all server stuff wich can put data in MySQL > > > for its brother, php web. > > > > > > http://cyrusimap.web.cmu.edu/imapd/man/notifyd.8.html > > > http://pwet.fr/man/linux/commandes/zephyr > > > > > > M. > > > > > > > > > > > > > > > -- > > Brennan Stehling > > http://brennan.offwhite.net/blog/ > > http://www.smallsharptools.com/ > > > > > > > > > -- > Till Klampaeckel > e: mailto:klimpong@gmail.com <mailto:klimpong@gmail.com> > p: +491704018676 > l: http://beta.plazes.com/whereis/till <http://beta.plazes.com/whereis/till> > > Want to know what's up in Berlin? > - http://berlin.metblogs.com > -- Brennan Stehling http://brennan.offwhite.net/blog/ http://www.smallsharptools.com/
Yes, it is not exactly a straightforward solution. I would like a more modular mail system which would give us more hooks to handle incoming mail. I know you can create milters, etc, but those are not a trivial thing to build. I once created extensions to the Apache web server with mod_perl which was really quite easy and I was able to make Apache do something the available extensions could not do alone. (SSL secured reverse proxy with LDAP lookup)
There is the James mail server which was created with Java which I believe had plans for plugins. But I cannot really run Java on my FreeBSD 5.x server.
Is there a modular IMAP server which has hooks for languages like Perl, Python or Ruby?
java for mail server is just a joke. Keep java for java stuff.
You can use existing product for that. The lowest level is SMTP
piping. For optimisation, you can use LMTP, the local version of
SMTP. If you're perl coder, just use Amavis.
In my configuration, i use Postfix, then amavis and after Cyrus.
Amavis do cleaning stuff, like virus search with ClamAV, domain
trashing with pyzor, spam hunting with spamassassin.
You can add your stuff here, amavis will ask you when a mail is
coming. Milter, the Sendmail interface is now coming to Postfix guy.
There is a python-milter.
With this filters you can alter mail content, or just tag it like
spamassassin do, after, mail can be redirected in the right mailbox,
with sieve rules, fetchmail rules or client side rules.
Just choose where you need to act.
M.
-
Arnór Heiðar -
Brennan Stehling -
Brett Patterson -
Eric Stadtherr -
Mathieu Lecarme -
Rob Smith -
till