Since this thread is of great interest to me, I want to pipe up here to ensure that not too much weight is put on the Received-header.
Harald Reindl suggested to grab the recipient from the Received: headers instead (rather than from "Delivered-To": or "Envelope-to:"), and I second that.
The Received-header does not necessarily contain the e-mail address that the message was delivered to. And even when it does, it may contain someone else's address entirely. Since the primary purpose of the "Received:"-header is to track the messages route as it passes from one mail server to the next, the recording of the destination e-mail address has been severely restricted. See:
http://tools.ietf.org/html/rfc5321#section-4.4
Specifcally, only one recipient address may be mentioned in a Received-header. And that is entirely optional.
Assuming the recipient's mail server is used by more than one party on a mailing list, and the previous mail relay submits the message in a single transaction, and the recipient's mail server includes the so-called "for clause", the e-mail address of exactly one of those parties will be in the "Received:"-header, and that just might be an address that is different from the recipient currently looking at it. Often, though, the for clause is just left out.
Matching identities is a creative endeavor by design. And taking advantage of fields offered by MTAs that are dedicated to informing mail clients about what the final delivery address was seems a more reliable way to go (i.e.: Envelope-To and Delivered-To). Currently, there is no RFC that prescribes a field that can be reliably used for the purpose of matching identities to the final delivery address of an e-mail.
Personally, I would suggest looking at all fields known to be of value for this purpose, ordered by reliability (including Envelope-To, Delivered-To, and Received, in that order).
Cheers, Sebastian