lists at pingle.org
Thu Dec 21 04:08:18 CET 2006
Brennan Stehling wrote:
> You can learn more about the ticket reports here...
> I think the voting idea would work but we can simply put up a vote on specific
> issues occasionally. On Apache projects they do it simply by posting an email
> to the list and letting everyone vote on each item. The team lead keeps track
> of it and adjust the priorities based on the results. That way it stays within
> the team.
What I had in mind was to use the voting as a means to measure how much of
the RoundCube user-base is affected by a given ticket. The periodic posting
may work well for feature requests, but keeping the voting within the team
doesn't necessarily help with checking how widespread a problem is with the
> I do like the idea of a few test environments. I think we can do it all on the
> same server. It just needs a separate website per install with a separate db
> for each one.
One server may be enough, but some situations may require some odd
configurations to make that work. I'd prefer to minimize the customization
needed, and to keep it closer to a standard user install. It might be
possible to do using jails or just multiple IPs. However, when you start
talking about all of the configuration possibilities, obviously there would
need to be limits as to just how far we'd have to go. Apache (1.3.x, 2.x),
PHP (4.x, 5.x), MySQL (4.x, 5.x), Postgres, not to mention IMAP servers:
Courier, Dovecot, etc, etc. It could get out of hand fast no matter how it
is done. :) The most popular are the most important though (Apache 2, PHP 5,
MySQL 4, etc)
> I have a FreeBSD server and could set up a test environment.
I have quite a few active servers at work (~20), and a lot more sitting
around collecting dust. Most of the ones collecting dust are at least dual
CPU PIII-800s with a GB or two of ram. Not speed demons by today's
standards, but for what needs done here it wouldn't have to be fast.
> Perhaps you could help me with an automatic provisioning system to deploy
> updates daily or weekly and automate some tests using Selenium. Dropping the db
> and rebuilding it with the latest update, generating a series of test emails and
> loading up the Selenium tests would not be all that difficult. And that does
> not require programming experience, but with your SA background you would know
> what to do.
I haven't messed with Selenium at all, but getting the backends in place
shouldn't be that hard. If you're just talking about getting a current SVN
revision in a directory and wiping/reloading the DB, that's really easy.
The test e-mail would be a little tougher to do, but still not hard. Copying
the messages into the right place is easy, but coming up with all of the
test cases might be rough. There are a lot of test cases that will need to
be considered (Plain vs HTML, MIME/base64, Attachments of all shapes and
sizes and each of those cases from multiple clients.)
> What I do not know yet is how to run the Selenium scripts on a server like
> FreeBSD or Linux. I suppose I could have a job run on a Windows server which
> has FireFox and MSIE installed so it can run the Selenium scripts. Of course
> someone could start the Selenium tests manually.
Because Selenium is a UI testing package, I'm not sure if there is a way to
automate it on a remote box unless you have VNC access or something similar,
because the tests run inside a browser. It might be best to have tests for
all platforms, though I do not have access to any Mac hardware. (And either
their page is out of date or their browser support is really lagging... Only
Opera 8 and Firefox 1.5?)
> But I also do not have time till mid January.
I'm not even sure when my schedule will clear up, but at this rate, probably
early January. I still find time now and then to poke around the trouble
tickets, I just wish I was more familiar with the code so I could jump in
and help out more directly. Coding isn't a problem, it's knowing where to
look and the how it all flows, which takes time to get into.
And of course you also have to consider the delicate balance where all of
this extra testing and overhead stops being helpful and starts being a
burden to developers. That's where making it quick, easy, and painless would
really help. The last thing we'd want is to discourage someone from coding
because they don't want to go through all the "trouble" of testing... (I'm
as guilty of putting off testing because of that as anyone!)
More information about the Dev