By default RC only polls the subscribed folders. So the rest is pretty much ignored. At least that should be the case.
Which IMAP server do you use? Does it sit on the same server, or remote, etc.
I use Dovecot 0.99, and the IMAP server is on the same server as Roundcube. I don't see any speed problems or high CPU utilization when I use another PHP-based webmail product (Ilohamail).
Doug