Regular expressions for breaklines (regex)

[atDev]

Where in the files/code are the breaklines checked? I'll code it in.

[atDev]

Nevermind... for anyone interested here yah go.

When entering the breakline make sure it begins with / as this triggers it to do a regex compare.

/modules/parser/functions_parsercore.php

Replace:

function processBreakLines($content)
{
    global $_SWIFT;
    parserDebug("NOTICE: Processing Breaklines");

    $breakline = &$_SWIFT["breaklinecache"];

    parserDebug("NOTICE: Breakline Total: ".count($breakline));
    if (!is_array($breakline))
    {
        parserDebug("NOTICE: No Breaklines found!");
        return $content;
    }

    foreach ($breakline as $key=>$val)
    {
        parserDebug("NOTICE: Processing Breakline: ".$val);
        
        $breakcontent = reversestrchr($content, $val);

        if ($breakcontent)
        {
            parserDebug("NOTICE: Breakline Triggered!");
            // Eureaka! Eureaka! Eureaka!
            return $breakcontent;
        }
    }

    return $content;
}

With:

function processBreakLines($content)
{
    global $_SWIFT;
    parserDebug("NOTICE: Processing Breaklines");

    $breakline = &$_SWIFT["breaklinecache"];

    parserDebug("NOTICE: Breakline Total: ".count($breakline));
    if (!is_array($breakline))
    {
        parserDebug("NOTICE: No Breaklines found!");
        return $content;
    }

    foreach ($breakline as $key=>$val)
    {
        parserDebug("NOTICE: Processing Breakline: ".$val);
        if(substr($val,0,1) == '/') {
            $splits = preg_split($val,$content);
            $breakcontent = count($splits) ? $splits[0] : false;   
        } else {
            $breakcontent = reversestrchr($content, $val);
        }
        if ($breakcontent)
        {
            parserDebug("NOTICE: Breakline Triggered!");
            // Eureaka! Eureaka! Eureaka!
            return $breakcontent;
        }
    }

    return $content;
}

Note: see bug fix in post Regular expressions for breaklines (regex)

[Jamie Edwards]

Hi atDev,

So this modification will work with and without regular expressions entered as breaklines?

[atDev]

So this modification will work with and without regular expressions entered as breaklines?

Yes. The main issue to look at is how to determine if the breakline entered in the kayako admin area should be treated as regex or not.

There are two ways to do this in my opinion.

1. Check some character in the breakline that signifies it as regex, like I did above. We check the first character, if its a /, we know this is a regex string.

2. Do a regex no matter what. Since all the kayako code is doing right now is finding the breakline and selecting all text before it, we can do this all at once with regex.

I have tested #1, as seen in the code above.

For #2 it would be like:

function processBreakLines($content)
{
    global $_SWIFT;
    parserDebug("NOTICE: Processing Breaklines");

    $breakline = &$_SWIFT["breaklinecache"];

    parserDebug("NOTICE: Breakline Total: ".count($breakline));
    if (!is_array($breakline))
    {
        parserDebug("NOTICE: No Breaklines found!");
        return $content;
    }

    foreach ($breakline as $key=>$val)
    {
        parserDebug("NOTICE: Processing Breakline: ".$val);
        $splits = preg_split('/'.$val.'/',$content);
        $breakcontent = count($splits) ? $splits[0] : false;   

        if ($breakcontent)
        {
            parserDebug("NOTICE: Breakline Triggered!");
            // Eureaka! Eureaka! Eureaka!
            return $breakcontent;
        }
    }

    return $content;
}

Keep in mind with this code, you no longer need to enter / and / before and after the regex string as we add this anyway in the above code.

A common reply line is something like:
On 12/17/07, Some Name <test@test.com> wrote:

The regex code below will work for this (has been tested):
/On [0-9]{1,2}\/[0-9]{1,2}\/[0-9]{1,2},(.)+ <(.)+@(.)+> wrote:/

Keep in mind again, the above works for the first solution given, if you use the code in this post (which is untested) you would remove the / and / like:
On [0-9]{1,2}\/[0-9]{1,2}\/[0-9]{1,2},(.)+ <(.)+@(.)+> wrote:

[atDev]

As far as implementing this in the core kayako code, I would tell your developers to use solution #2 as you are going to be comparing a string versus the email content no matter what.

preg_split will work even if its given a static string (no regex) but it may be a bit slower.

It is either that or do an if/else statement like I did in #1 solution above.

[Jamie Edwards]

Hi atDev,

Thanks a lot for sharing this - we will be implementing a regex solution soon. I have split your posts to a new thread here.

[atDev]

I did some basic tests on my own and everything parsed smoothly, I will update this thread if any noticeable problems pop up but its a pretty basic change so no side effects should be seen.

[atDev]

For anyone needing them. After reviewing some more emails not getting parsed we have added these three regex lines to catch variations of the breakline:

/On [0-9]{1,2}\/[0-9]{1,2}\/[0-9]{1,2},(.)+ <(.)+@(.)+> wrote:/

/On [A-Za-z]{3} [0-9]{1,2}, at [0-9]{1,2}:[0-9]{2} (PM|AM), (.)+ wrote:/

/On [A-Za-z]{3} [0-9]{1,2}, [0-9]{4} [0-9]{1,2}:[0-9]{2} (PM|AM), (.)+ <(.)+@(.)+> wrote:/

[atDev]

A bugfix. It appears preg_split returns the entire string if no splits are done, which in theory it should. This causes it to no parse/run through the rest of the breaklines. I switched to using this:

    foreach ($breakline as $key=>$val)
    {
        parserDebug("NOTICE: Processing Breakline: ".$val);
        if(substr($val,0,1) == '/') {
            $breakcontent = preg_match($val, $content, $matches, PREG_OFFSET_CAPTURE) ? substr($content,0,$matches[0][1]) : false;   
        } else {
            $breakcontent = reversestrchr($content, $val);
        }
        if ($breakcontent)
        {
            parserDebug("NOTICE: Breakline Triggered!");
            // Eureaka! Eureaka! Eureaka!
            return $breakcontent;
        }
    }

This uses preg_match instead which is a surefire way of true/false match/no match instead of splitting it. Then we just do a substr using the offset of the first match, much like the script already does but it uses static string matching.

[maykelrr]

Hi, I would like to implement this in my support site, but I have a leased eSupport and the source is zend-encoded, is there any way to obtain the same zend-encoded file with this change on it?

Thanks!
Maykel Rodriguez
SpeedyRails - Celebrating our First Anniversary!

[craigbrass]

You need to contact Jamie (jamie.edwards<!at!>kayako<!dot!>com) listing the files you need and he will be able to provide.

[Jamie Edwards]

Hi Maykel,

Unfortunatly the open files are unable on your leased license; in order to have near-unrestricted access to the source code (except for the licensing functions), you need to have an owned license.

[craigbrass]

I thought you were able to request individual files and get them unencoded?

[Ryan Lederman]

I've committed this feature to CVS as of today. Breaklines that are to be treated as a regular expression start with "regex:" and must contain the start/stop symbols of a normal regex; (e.g. "regex:@[abcd]@")