The "parser.lockfile" is blocking POP3/IMAP fetching

[milksama]

We are also encountering this problem with our setup.

It is fortunate to see this has been addressed (or that the lock file is automatically removed after a given period), though we still have a concern:

This problem with the introduction of the lock file occurred near the beginning of a holiday weekend here. Although we are fetching email now, the "Posted on:" field (within the ticket view) has an incorrect time stamp -- using the date parsed rather than the date submitted. Is there a setting to have tickets instead use the date on which the email was submitted rather than have it as it is now, with 1000 emails apparently submitted simultaneously? If not, could such a feature be introduced? For a setup that better addresses our customers' needs, it would really be preferable to time stamp the date a given email was submitted, not parsed.

Also, around what time might 3.20.03 be available?

Thank you.


Mike

[Ryan Lederman]

DO NOT FOLLOW THESE INSTRUCTIONS ANYMORE. THEY ARE NO LONGER CORRECT.


If you are experiencing problems with the parser lock file that are disrupting business, you can apply the following patch to your desk instead of waiting for 3.20.03 (which, by the way, will likely be called 3.30.00)

Take the following files from the latest CVS build and replace the ones on your server with them (make sure to take a backup in case it does not work):

/modules/parser/cron_parser.php
/locale/en-us/emailparser.php

These files also address bug #463 (parser.lock file should automatically expire)

[milksama]

Ryan: Great! Thank you. I will have these updated.

[Geert-Jan]

Originally Posted by Ryan Lederman

If you are experiencing problems with the parser lock file that are disrupting business, you can apply the following patch to your desk instead of waiting for 3.20.03 (which, by the way, will likely be called 3.30.00)

Take the following files from the latest CVS build and replace the ones on your server with them (make sure to take a backup in case it does not work):

/modules/parser/cron_parser.php
/locale/en-us/emailparser.php

These files also address bug #463 (parser.lock file should automatically expire)

Just uploaded the two files, and check the next few days.

In our case:
- v 3.20.02 Zend
- IMAP
- PHP5.2.6
- 2nd esupport, not yet in production yet..

[jbsolutios]

Hi Ryan

When I installed the 2 files you recommended on to our production system (3.20.02) from Version: 3.20.03 (CVS) - 10 Jun 2008 06:10 AM, I got the following error: -

Warning: require_once(./modules/parser/functions_parser_loopcontrol.php) [function.require-once]: failed to open stream: No such file or directory in C:\Inetpub\http://www.help.solutios.com\modules...ron_parser.php on line 129

Fatal error: require_once() [function.require]: Failed opening required './modules/parser/functions_parser_loopcontrol.php' (include_path='.;c:\php\includes') in C:\Inetpub\http://www.help.solutios.com\modules...ron_parser.php on line 129

I then installed the following from CVS: -

/modules/functions_parser_loopcontrol.php

When I did this, emails were imported, but I got the following error: -

Total messages in inbox: 1
Fetching message #1
Account is IMAP
Fetching message UID: 2042
Invalid SQL: select length from swemailloopthreshholds order by length desc limit 1 (Table 'kayako_suite.swemailloopthreshholds' doesn't exist)Invalid SQL: delete from swemailloopcontacttimes where contacttime < 1213098370 (Table 'kayako_suite.swemailloopcontacttimes' doesn't exist)Invalid SQL: delete from swemailloopcutsinplace where restoretime < 1213098370 (Table 'kayako_suite.swemailloopcutsinplace' doesn't exist)Invalid SQL: select count(*) from swemailloopcutsinplace where address like ' jbredcar@mac.com: John Benson ' (Table 'kayako_suite.swemailloopcutsinplace' doesn't exist)Invalid SQL: insert into swemailloopcontacttimes(address, contacttime) values(' jbredcar@mac.com: John Benson ', 1213098370); (Table 'kayako_suite.swemailloopcontacttimes' doesn't exist)Invalid SQL: select contacttime from swemailloopcontacttimes where address like ' jbredcar@mac.com: John Benson ' (Table 'kayako_suite.swemailloopcontacttimes' doesn't exist)Invalid SQL: select length, maxhits, restoreafter from swemailloopthreshholds (Table 'kayako_suite.swemailloopthreshholds' doesn't exist)Total messages in inbox: 0
0 Total messages
Rejected messages: 0
Accepted messages: 0

Many thanks

JB

[John Haugeland]

DO NOT FOLLOW THESE INSTRUCTIONS ANYMORE. THEY ARE NO LONGER CORRECT.

Originally Posted by jbsolutios

When I installed the 2 files you recommended on to our production system (3.20.02) from Version: 3.20.03 (CVS) - 10 Jun 2008 06:10 AM, I got the following error: -

Sorry. This is one of the risks of hybridizing systems, and one of the reasons we try not to do it when possible. You've managed to acquire about one quarter of the code involved in the new loop cutter system, a defense against a more generic class of those 10-million-in-a-row loops that happen once every blue moon.

That new system is dependant on the 3.30 upgrade script putting several new tables into the database and populating one of them with defaults. It is, unfortunately, not an option to upgrade you to 3.30 now, as we still have a few different tables to add for a different thing, and it would just be a mess.

What I'm going to advise you to do is to comment out two lines in that file, which should effectively disable the loop cutting system. When next stable comes out, please upgrade to get a unified tested release.





On line 129, you should see this line:

require_once ("./modules/parser/functions_parser_loopcontrol.php");

Please comment it out by placing two forward slashes and a space before that line, as follows:

// require_once ("./modules/parser/functions_parser_loopcontrol.php");

Also, on line 234, you should see the following line:

$isLoopBlocked = StartLoopBlockCheck($fixedheader);

Comment it out, and then write $isLoopBlocked = false; on the next line (must not be on the same line!), as follows:

//              $isLoopBlocked = StartLoopBlockCheck($fixedheader);     
                $isLoopBlocked = false;

Those two modifications should moot the loop cutter until you can upgrade to a unified release.

[jbsolutios]

Dear John

Many thanks for your prompt reply! We have never used any CVS files before for this very reason, but are finding that we have stale lock files at least once per day.

For the help of others, the modifications above need to be carried out in: -

/modules/parser/cron_parser.php

Regards

John

[John Haugeland]

Originally Posted by jbsolutios

We have never used any CVS files before for this very reason

A wise long-term policy. Kayako is making effort to move to shorter release cycles, to reduce the reason to want to do things like this.

As far as the fast reply, it's Ryan you should thank, not me; he pointed this thread out to me.

[Geert-Jan]

Originally Posted by John Haugeland

Sorry. This is one of the risks of hybridizing systems, and one of the reasons we try not to do it when possible. You've managed to acquire about one quarter of the code involved in the new loop cutter system, a defense against a more generic class of those 10-million-in-a-row loops that happen once every blue moon.

That new system is dependant on the 3.30 upgrade script putting several new tables into the database and populating one of them with defaults. It is, unfortunately, not an option to upgrade you to 3.30 now, as we still have a few different tables to add for a different thing, and it would just be a mess.

What I'm going to advise you to do is to comment out two lines in that file, which should effectively disable the loop cutting system. When next stable comes out, please upgrade to get a unified tested release.





On line 129, you should see this line:

require_once ("./modules/parser/functions_parser_loopcontrol.php");

Please comment it out by placing two forward slashes and a space before that line, as follows:

// require_once ("./modules/parser/functions_parser_loopcontrol.php");

Also, on line 234, you should see the following line:

$isLoopBlocked = StartLoopBlockCheck($fixedheader);

Comment it out, and then write $isLoopBlocked = false; on the next line (must not be on the same line!), as follows:

//              $isLoopBlocked = StartLoopBlockCheck($fixedheader);     
                $isLoopBlocked = false;

Those two modifications should moot the loop cutter until you can upgrade to a unified release.

John,

You want this on the 20.02 stable file or on the 20.03 cvs file?? We still got the problem whit the 20.03 cvs files...

Thanks.

[John Haugeland]

Originally Posted by Geert-Jan

You want this on the 20.02 stable file or on the 20.03 cvs file?? We still got the problem whit the 20.03 cvs files...

The fix didn't exist until the repository was renamed to 3.30.00. What you have is out of date and does not have the required code.

[Geert-Jan]

Yep, i've seen. But what wil the solution to fix this issue then..???

[GoneShootin]

Yeah but WHEN will it be released?


I'm kidding I'm kidding!

[FrightFactor]

I just ran into the same problem.

I appreciate that you're taking your time to make sure that everything is stable and works fine before you release any new Versions. And i encourage you to do so, great job btw.

but when it comes down to a point, where a lot of users complaining about a lockfile, which takes down the entire email communication of your company (if someone just uses the helpdesk), than you should at least stop working on your next Release and put a working fix together which eliminates the problem in 3.20.02 and not just putting it on the bug tracker and fix it with the next release. Or handing out files from a newer version which results into error messages.

just my opinion.

[GoneShootin]

On that.

What I ended up doing was educating the users. If they found that they didn't have any emails come in the morning, I'd get them to run the cron php file, and check for any link that say's "Delete lock file".

All with the promise that a new version of Kayako was on the way to resolve the issue.

I think educating the users is certainly the first step, especially if you can identify IT savvy ones within the group.

[Geert-Jan]

So, for the time being, can I make a cron whit

http://bla.bla.bla/cron/index.php?_t=parser&deletelockfile=1

@ every hour or is there a other temparary solution, downgrade to version 3.xx.xx?


Thanks.