Brennan Stehling wrote:
You can learn more about the ticket reports here... http://trac.roundcube.net./trac.cgi/wiki/TracReports
I think the voting idea would work but we can simply put up a vote on specific issues occasionally. On Apache projects they do it simply by posting an email to the list and letting everyone vote on each item. The team lead keeps track of it and adjust the priorities based on the results. That way it stays within the team.
What I had in mind was to use the voting as a means to measure how much of the RoundCube user-base is affected by a given ticket. The periodic posting may work well for feature requests, but keeping the voting within the team doesn't necessarily help with checking how widespread a problem is with the users.
I do like the idea of a few test environments. I think we can do it all on the same server. It just needs a separate website per install with a separate db for each one.
One server may be enough, but some situations may require some odd configurations to make that work. I'd prefer to minimize the customization needed, and to keep it closer to a standard user install. It might be possible to do using jails or just multiple IPs. However, when you start talking about all of the configuration possibilities, obviously there would need to be limits as to just how far we'd have to go. Apache (1.3.x, 2.x), PHP (4.x, 5.x), MySQL (4.x, 5.x), Postgres, not to mention IMAP servers: Courier, Dovecot, etc, etc. It could get out of hand fast no matter how it is done. :) The most popular are the most important though (Apache 2, PHP 5, MySQL 4, etc)
I have a FreeBSD server and could set up a test environment.
I have quite a few active servers at work (~20), and a lot more sitting around collecting dust. Most of the ones collecting dust are at least dual CPU PIII-800s with a GB or two of ram. Not speed demons by today's standards, but for what needs done here it wouldn't have to be fast.
Perhaps you could help me with an automatic provisioning system to deploy updates daily or weekly and automate some tests using Selenium. Dropping the db and rebuilding it with the latest update, generating a series of test emails and loading up the Selenium tests would not be all that difficult. And that does not require programming experience, but with your SA background you would know what to do.
I haven't messed with Selenium at all, but getting the backends in place shouldn't be that hard. If you're just talking about getting a current SVN revision in a directory and wiping/reloading the DB, that's really easy.
The test e-mail would be a little tougher to do, but still not hard. Copying the messages into the right place is easy, but coming up with all of the test cases might be rough. There are a lot of test cases that will need to be considered (Plain vs HTML, MIME/base64, Attachments of all shapes and sizes and each of those cases from multiple clients.)
What I do not know yet is how to run the Selenium scripts on a server like FreeBSD or Linux. I suppose I could have a job run on a Windows server which has FireFox and MSIE installed so it can run the Selenium scripts. Of course someone could start the Selenium tests manually.
Because Selenium is a UI testing package, I'm not sure if there is a way to automate it on a remote box unless you have VNC access or something similar, because the tests run inside a browser. It might be best to have tests for all platforms, though I do not have access to any Mac hardware. (And either their page is out of date or their browser support is really lagging... Only Opera 8 and Firefox 1.5?)
But I also do not have time till mid January.
I'm not even sure when my schedule will clear up, but at this rate, probably early January. I still find time now and then to poke around the trouble tickets, I just wish I was more familiar with the code so I could jump in and help out more directly. Coding isn't a problem, it's knowing where to look and the how it all flows, which takes time to get into.
And of course you also have to consider the delicate balance where all of this extra testing and overhead stops being helpful and starts being a burden to developers. That's where making it quick, easy, and painless would really help. The last thing we'd want is to discourage someone from coding because they don't want to go through all the "trouble" of testing... (I'm as guilty of putting off testing because of that as anyone!)
Jim