...still seems to be quite a strange thing in Roundcube.
Just did a search for the term "Ibiza" in the subject:
First of all, some messages even when searching in just one specific folder are not found at all. Although the subject clearly contains this string, a message with the subject "Auftragsbestätigung Düsseldorf-Valencia-Ibiza 12JUN16" is not being found. Looking at the source of the message, it becomes quite clear, why:
Subject:
=?UTF-8?Q?Auftragsbest=C3=A4tigung_D=C3=BCsseldorf-Valencia-Ibiz?=
=?UTF-8?Q?a_12JUN16?=
See...? The subject is folded right before the last character of "Ibiza", and the search routine apparently doesn't unfold such headers. Not good.
OTOH, a search for the same string "Ibiza" across ALL folders does find messages which do not contain this string in the subject at all and therefore shouldn't be found:
"Re: Anfrage Mexiko-Miami 02DEC16 und Montréal-New York 10DEC16" "Re: IBZ - Vigo" "✈ Aktuelle Flugzeugverfügbarkeiten" "Re: EMPTY LEG C650 (CITATION III ) Availability" "Re: ZRH - DUS"
Just a few examples of HUNDREDS of (wrong) matches. What's the search routine looking for? Apparently not only for "Ibiza", but for what else?
Indeed, those messages DO contain the string "Ibiza" in the body, but I didn't ask for a search in the subject AND the body.
I still may have more issues with regards to searching messages which I did report last year already, but that should be it for today.
Searching for messages is quite critical for me, as I'm using Roundcube not (only) for private purposes - as you may see above.
Cheers,
Michael Heydekamp Co-Admin freexp.de Düsseldorf/Germany
On 06/08/2016 02:07 AM, Michael Heydekamp wrote:
Indeed, those messages DO contain the string "Ibiza" in the body, but I didn't ask for a search in the subject AND the body.
These may be IMAP server issues. Enable imap_debug to see what is the query and what is the result.
On 16-06-07 10:27 PM, A.L.E.C wrote:
On 06/08/2016 02:07 AM, Michael Heydekamp wrote:
Indeed, those messages DO contain the string "Ibiza" in the body, but I didn't ask for a search in the subject AND the body.
These may be IMAP server issues. Enable imap_debug to see what is the query and what is the result.
to elaborate on what alec said, roundcube uses the imap SEARCH command to find messages - perhaps your imap server isn't searching in UTF7IMAP encoded strings correctly (or at all).
i have a folder here with two messages (with subjects matching yours, including a wrapped line for the subject):
Maildir/.myfolder/cur% zgrep -A1 Subject * 1465399407.P44335Q0M64750.smtpin01,S=34512,W=35061:2,:Subject: ibiza party time 1465399407.P44335Q0M64750.smtpin01,S=34512,W=35061:2,:X-Spamscore: 0 1465399409.P45445Q0M30546.smtpin01,S=15699,W=15955:2,:Subject: =?UTF-8?Q?Auftragsbest=C3=A4tigung_D=C3=BCsseldorf-Valencia-Ibiz?= 1465399409.P45445Q0M30546.smtpin01,S=15699,W=15955:2,: =?UTF-8?Q?a_12JUN16?=
and the imap server returns both of them in response to a search (with either UTF-8 or ASCII):
a select myfolder
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft *)]
Flags permitted.
- 2 EXISTS
- 0 RECENT
- OK [UNSEEN 1] First unseen.
- OK [UIDVALIDITY 1430938009] UIDs valid
- OK [UIDNEXT 3] Predicted next UID
- OK [HIGHESTMODSEQ 1] Highest
a OK [READ-WRITE] Select completed (0.003 secs). a01 SEARCH CHARSET UTF-8 SUBJECT "ibiza"
- SEARCH 1 2
a01 OK Search completed (0.002 secs). a02 SEARCH CHARSET US-ASCII SUBJECT "ibiza"
- SEARCH 1 2
a02 OK Search completed (0.000 secs). a01 SEARCH CHARSET UTF-8 SUBJECT "Auftragsbest"
- SEARCH 2
a01 OK Search completed (0.000 secs).
if your imap server doesn't support searching in utf7imap encoded strings, it'd pretty hard for roundcube to work around that, considering that there are a huge number of ways that string could have been encoded (depending on the charset it was encoded from). roundcube would need to issue an imap search for every possibility, which isn't reasonable (and would be incredibly slow, even if it were done).
On 06/08/2016 09:04 PM, Brendan wrote:
to elaborate on what alec said, roundcube uses the imap SEARCH command to find messages - perhaps your imap server isn't searching in UTF7IMAP encoded strings correctly (or at all).
Brendan, thanks for the explanation, but just to be clear, UTF-7 encoding has nothing to do with searching. This encoding is used only for mailbox names.
Am 08.06.2016 21:04, schrieb Brendan:
On 16-06-07 10:27 PM, A.L.E.C wrote:
On 06/08/2016 02:07 AM, Michael Heydekamp wrote:
Indeed, those messages DO contain the string "Ibiza" in the body, but I didn't ask for a search in the subject AND the body.
These may be IMAP server issues. Enable imap_debug to see what is the query and what is the result.
to elaborate on what alec said, roundcube uses the imap SEARCH command to find messages - perhaps your imap server isn't searching in UTF7IMAP encoded strings correctly (or at all).
Dunno why you're bringing up an UTF7-issue? The subject we're talking about was a) UTF8-encoded, and b) the encoding is not the issue at all. The issue is the FOLDED subject.
if your imap server doesn't support searching in utf7imap encoded strings, it'd pretty hard for roundcube to work around that, considering that there are a huge number of ways that string could have been encoded (depending on the charset it was encoded from). roundcube would need to issue an imap search for every possibility, which isn't reasonable (and would be incredibly slow, even if it were done).
Sorry, I don't get what you're trying to point at. I'm just saying that Roundcube should unfold subjects (and other headers) before it checks if they are matching a particular search. Currently it seems that it doesn't do that.
Alec's response above is BTW not referring to this issue at all. He's referring to item 2) of my initial message, which describes a totally different problem.
Cheers,
Michael Heydekamp Co-Admin freexp.de Düsseldorf/Germany
On 16-06-10 02:20 PM, Michael Heydekamp wrote:
Am 08.06.2016 21:04, schrieb Brendan:
On 16-06-07 10:27 PM, A.L.E.C wrote:
On 06/08/2016 02:07 AM, Michael Heydekamp wrote:
Indeed, those messages DO contain the string "Ibiza" in the body, but I didn't ask for a search in the subject AND the body.
These may be IMAP server issues. Enable imap_debug to see what is the query and what is the result.
to elaborate on what alec said, roundcube uses the imap SEARCH command to find messages - perhaps your imap server isn't searching in UTF7IMAP encoded strings correctly (or at all).
Dunno why you're bringing up an UTF7-issue? The subject we're talking about was a) UTF8-encoded, and b) the encoding is not the issue at all. The issue is the FOLDED subject.
as alec pointed out, i was incorrect in mentioning UTF7IMAP, the problem is with UTF8 (which is what i meant, i just typed the wrong encoding method both times in the message).
if your imap server doesn't support searching in utf7imap encoded strings, it'd pretty hard for roundcube to work around that, considering that there are a huge number of ways that string could have been encoded (depending on the charset it was encoded from). roundcube would need to issue an imap search for every possibility, which isn't reasonable (and would be incredibly slow, even if it were done).
Sorry, I don't get what you're trying to point at. I'm just saying that Roundcube should unfold subjects (and other headers) before it checks if they are matching a particular search. Currently it seems that it doesn't do that.
roundcube calls your IMAP server for search (with those imap search commands i had performed manually). roundcube isn't responsible for unfolding the header during the search - your imap server is.
Alec's response above is BTW not referring to this issue at all. He's referring to item 2) of my initial message, which describes a totally different problem.
oh, probably. the problem of it not matching that utf8 folded subject is more interesting though. debug for the
going back to the folded subject one:
Subject:
=?UTF-8?Q?Auftragsbest=C3=A4tigung_D=C3=BCsseldorf-Valencia-Ibiz?=
=?UTF-8?Q?a_12JUN16?=
roundcube asks the imap server to do a search for messages matching that subject. if the imap server does not respond with this message in response to that search, then your imap server is either not unfolding the subject headers or not converting the subject into UTF8 before trying to match against the subject.
if you telnet to the imap server and log in, you should be able to tell which of the two it is:
a01 SEARCH CHARSET UTF-8 SUBJECT "est=C3=A4"
- if this returns the message, then the imap server is not converting
the subject to UTF-8 when trying to search (if it were, the =C3=A4 bit of utf-8 encoding would not match)
a01 SEARCH CHARSET UTF-8 SUBJECT "Düsse"
- this also ought to return the message, but wouldn't if the imap server
was failing to convert the subject to UTF-8 while searching
a01 SEARCH CHARSET UTF-8 SUBJECT "12JUN16"
- if this does not return the message, then the imap server is not
folding the headers correctly when trying to search (since that 12JUN16 string is on the folder part of the header)
a01 SEARCH CHARSET UTF-8 SUBJECT "Ibiza"
- ought to return the message. but is apparently not (my imap server
does return both messages, and roundcube displays both as a result).
again, here's my imap server responding to those queries correctly (message ID 2 has the same subject you reported above - the "a" utf8 encoded and the subject line wrapped, MID 1 is a message with Ibiza in the subject, but not wrapped or utf8 encoded):
correctly finding the message: a01 SEARCH CHARSET UTF-8 SUBJECT "ibiza"
- SEARCH 1 2
a01 OK Search completed (0.000 secs).
correctly not finding the utf8 encoding bits: a01 SEARCH CHARSET UTF-8 SUBJECT "est=C3=A4"
- SEARCH
a01 OK Search completed (0.000 secs).
correctly finding the message with the wrapped part of the subject: a01 SEARCH CHARSET UTF-8 SUBJECT "12JUN16"
- SEARCH 1
a01 OK Search completed (0.000 secs).
correctly finding a utf8 string in the subject part that isn't folded: a01 SEARCH CHARSET UTF-8 SUBJECT "Düsse"
- SEARCH 1
a01 OK Search completed (0.000 secs).
Am 08.06.2016 07:27, schrieb A.L.E.C:
On 06/08/2016 02:07 AM, Michael Heydekamp wrote:
Indeed, those messages DO contain the string "Ibiza" in the body, but I didn't ask for a search in the subject AND the body.
These may be IMAP server issues. Enable imap_debug to see what is the query and what is the result.
And I think the scope of Roundcube when searching across ALL folders is simply not correct (on all folders being searched).
Bt I'll check what imap_debug will show.
And what about item 1) of my previous message (folded subject)? No response to that one...?
Cheers,
Michael Heydekamp Co-Admin freexp.de Düsseldorf/Germany
On 06/10/2016 11:10 PM, Michael Heydekamp wrote:
And what about item 1) of my previous message (folded subject)? No response to that one...?
Both issues are likely your IMAP server issues. Use debug to see what is send by Roundcube to the server.
-
A.L.E.C -
Brendan -
Michael Heydekamp