How to increase imap fetch frequency

[jrp2]

Hi,

Sorry if this is a stupid newbie question. I am just giving SupportSuite a try.

I am doing imap fetching for email retrieval. Looking at the scheduled tasks, it is supposed to be doing a fetch every 10 minutes. I would much prefer 5 minutes.

I am running a wget in cron to /cron/index.php every 5 minutes.

So, how do I modify the scheduled task settings? I can't see any way to modify these. Are they hardcoded?

Any tips or guidance is appreciated.

Regards,

JRP2

PS. Yes, I am aware of the risk of doing too frequent fetches, as they may overlap. My mail server is on the same 1 gig switch as the SupportSuite server, and both machines are fast, and my email volume is quite low (only a few new tickets a day, maybe a total of 5-20 messages). I also reduced the number of emails per pass to 5 from 20, just in case.

SupportSuite 3.20.02 Stable
Ubuntu 7.10 LAMP server
PHP 5.2.3
MySQL 5.0.45

[Jamie Edwards]

Hi JRP2,

Yes, the fetch interval is hard coded and cannot be changed from 10 minutes without modifying the code.

[craigbrass]

That is incorrect Jamie. It is defined in database table swcron. Simply look for the one with name "parser" and change cminute to what you want.

[Jamie Edwards]

Originally Posted by craigbrass

That is incorrect Jamie. It is defined in database table swcron. Simply look for the one with name "parser" and change cminute to what you want.

For all intents and purposes, it is hardcoded in that you cannot modify it via the interface; but you are right that it can be changed in the database.

[jrp2]

Originally Posted by craigbrass

That is incorrect Jamie. It is defined in database table swcron. Simply look for the one with name "parser" and change cminute to what you want.

Thank you both. I totally missed swcron when perusing the db.

Now, still having a problem that I cannot completely grok. I made the change about an hour ago, and have been watching it. It still seems to fetch every 10, sometimes 15 or 20, minutes. What appears to happen, is it only does one task every cron poll. The 15 minute interval seems to happen when the module actions runs, 20 on the hour when both module actions and hourly cleanup runs!

So, with this command being run by my system cron every 5 minutes:

/usr/bin/wget -O - --quiet "http://localhost/support/cron/index.php"

And these settings in the db (pasted from the SS Manage Tasks screen):

Hourly System Cleanup 1 Hour(s)
Daily System Cleanup 1 Day(s)
Ticket Module Actions 30 Minute(s)
Ticket Follow-up 10 Minute(s)
POP3/IMAP Fetching 5 Minute(s)
Mail Parser Cleanup 1 Day(s)

I am seeing this in the task log:

POP3/IMAP Fetching 10 Apr 2008 10:15 AM (20 minutes!)
Ticket Module Actions 10 Apr 2008 10:10 AM
Ticket Follow-up 10 Apr 2008 10:05 AM
Hourly System Cleanup 10 Apr 2008 10:00 AM
POP3/IMAP Fetching 10 Apr 2008 09:55 AM (10 minutes)
Ticket Follow-up 10 Apr 2008 09:50 AM
POP3/IMAP Fetching 10 Apr 2008 09:45 AM (15 minutes)
Ticket Follow-up 10 Apr 2008 09:40 AM
Ticket Module Actions 10 Apr 2008 09:35 AM
POP3/IMAP Fetching 10 Apr 2008 09:30 AM (10 minutes)
Ticket Follow-up 10 Apr 2008 09:25 AM
POP3/IMAP Fetching 10 Apr 2008 09:20 AM (15 minutes)
Ticket Follow-up 10 Apr 2008 09:15 AM
Ticket Module Actions 10 Apr 2008 09:10 AM
POP3/IMAP Fetching 10 Apr 2008 09:05 AM (15 minutes)
Hourly System Cleanup 10 Apr 2008 09:00 AM
Ticket Follow-up 10 Apr 2008 08:55 AM
POP3/IMAP Fetching 10 Apr 2008 08:50 AM (10 minutes)
Ticket Follow-up 10 Apr 2008 08:45 AM
POP3/IMAP Fetching 10 Apr 2008 08:40 AM

I am a little baffled by this. Seems like an overly paranoid safety mechanism or I am missing some setting.

I can see a couple possible solutions:

- Increase cron frequency to every 2 minutes or so.

- Decrease the frequency of some of the non-essential tasks like ticket followup and module actions.

And, perhaps a slight combination of the above.

I guess I could also do piping, but I don't like that for a couple of reasons.

Or, is there something I can do in cron, using parameters on the cron job (?ticket stuff, etc.) and more selectively controlling what is going on?

Or, is there some setting in the config to allow more than one task per poll interval?

Any thoughts or tips?

Thanks again!

JRP2

[jrp2]

Just to follow up. I did the following, and got it working pretty good (good enough):

• Changed parser frequency to 3 minutes.
• Backed off on ticket followup to 15 minutes
• Changed my system cron (wget /cron/index.php) to 3 minute intervals.

Also, and very important to prevent potential overlap issues, changed the number of emails to get per poll from 20 to 5.

With these changes I now get email polling every 3-12 minutes. Normally, it will be 3-6, sometimes 9, and rarely 12. One could probably make other changes like using odd, perhaps prime, numbers in the interval settings for each function to get it to minimize conflicts.

I will likely put in a bug on the overall problem I overcame. This could definitely be done better! Gotta say though, this was the only real problem I have run into while implementing Kayako, and it wasn't all that painful. Thanks again to Crag and Jamie for their prompt replies and useful pointers!

Disclaimers:

If you are going to make changes to your own system, be very careful. This is not configurable from the GUi for a reason. I would only make these changes if you know what you are doing, understand the problem, your email system is reliable, close (on same network in my case) and your email volume is reasonably low.

I am not a lawyer, your mileage may vary, and do not taunt happy fun ball!

[doug316]

It appears in my installation that the tasks run in more of a round-robin manner. When the cron job is fired off, it doesn't necessarily run the task that you would expect, and sometimes runs tasks that are not yet scheduled to run.

Am I missing something here? If I schedule a cron job for 10 minute intervals, I would expect my email to be POP'ed every 10 minutes. I believe I am seeing other tasks executing more often than they should, and that email is not checked more than once every few iterations.

[craigbrass]

Internal Crons / Scheduled Tasks are also fired when staff login to the staff / admin control panels.

[jrp2]

Originally Posted by craigbrass

Internal Crons / Scheduled Tasks are also fired when staff login to the staff / admin control panels.

That is just one more wrinkle in the slightly obscure workings of the cron processes in Kayako. It only slightly addresses Doug's comments.

I definitely agree with Doug that what you expect to happen, and what actually does happen, are rarely the same. I have observed this stuff fairly closely, and constantly seeing new and different things happening. It is rather bizarre at times.

What I have found, and my advice to anyone baffled by all this:

- Run your cron trigger app (wget under cron, or whatever) a bit more frequently than you think you should have to. Using defaults (no messing with the swcron table), I would run it every 3-4 minutes rather than every ten. That will likely get you a fetch every ten (mostly), and running the other tasks (ticket followup, hourly cleanup, etc.) during the other triggers. Do this, and you will barely even notice the oddities.

- Be patient! Once you have started the more frequent cron polls, wait about 90 minutes before you come to any conclusions. Things smooth out after then, after it has gone through a full 1 hour cycle of processes and caught up on backlogged tasks. Even then it won't be precisely what you expect, but it will be pretty close and likely good enough.

- DO NOT RELY ON SYSTEM ACTIVITY TO TRIGGER IT! Use a cron job to tickle the system for sure. If you don't use a cron trigger, and there is no one logged in overnight, it could take up a long time to get the first fetch in the morning as the system is trying to catch up on all the cleanup tasks it missed overnight.

Bottom line, this is one really bizarre aspect of Kayako. I really think the Kayako team should revisit this code and see how it could be done better. Though I understand some (not all) of the logic they are using, it is not very apparent what to expect and has some nasty side effects (very long intervals between fetches at times).

Hope this helps,

JP