How does v3 handle SPAM?

[nextcontact]

I wonder if an http://assp.sourceforge.net/ installation automatically protects all sub-systems like kayako ?

Take a look... i know i did and will be even happier if it integrates "by default" with kayako!

[Francisco]

Originally Posted by nextcontact

I wonder if an http://assp.sourceforge.net/ installation automatically protects all sub-systems like kayako ?

Take a look... i know i did and will be even happier if it integrates "by default" with kayako!

sweet, but it seems like a virtual SNMP server that will receive the connection from the host for e-mail and reject it or deny it based on white and black list, I like their stats though.

[Varun Shoor]

Hi,
I have made this thread a Sticky because I believe Spam prevention should form the base of any email management program/script out there.

I however would like to clarify some points on this:

1) Will there be Spam Prevention feature in v3.0? No, It wont make into Stable release but it is on the roadmap for v3.1. It will however not be simple bayesian filtering like our competitors have. Spam is ever evolving and simply tagging certain type of emails as Spam or Non Spam and then comparing keywords will never fix the issue. If it does for you, then you are one of the lucky ones. I receive almost 1500 spam emails daily at varun AT kayako.com, Bayesian filtering was tried and it worked only on 40% of emails. The thing that solved it? Black Lists and "Intelligent" checking by SpamAssasin coupled with Bayesian filtering. The spam is now down to 99.99% which is a big achievement in itself.

2) How will Spam Prevention be done in v3.1? As I said earlier it wont be simple Bayesian filtering but a bit more complex system. A system that carries checks like following on an incoming email:
What is the ratio of Misspelled words to correct spellings?
Do images appear in the HTML Part?
Is there any hidden text in HTML Email?
Does the email contain any links?

A lot of such checks will be combined with Bayesian filtering to form a scoring system akin to SpamAssasin does. This will be a true solution to combating spam, the solutions available right now are more of "gimmicks".

3) So why in v3.1? Why not in v3? This feature requires a lot of complex coding and will delay our v3 release further. We plan to release new builds every month with one major feature once v3 gold goes out.

4) What does Kayako recommend? We will ALWAYS recommend using Black Lists along with SpamAssasin to combat spam in the most efficient manner. People have suggested implementing Black List updating into Kayako but Blacklists are huge and will cause unnecessary overhead in the current product. There are Open Source products out there (SpamAssasin etc) that carry out spam filtering in the most optimal manner the added benefit is that they are tried and tested by millions of users.

Feedback as always is welcome.

Regards,

Varun Shoor

[Francisco]

Originally Posted by Varun [K]

Hi,
I have made this thread a Sticky because I believe Spam prevention should form the base of any email management program/script out there.

I however would like to clarify some points on this:

1) Will there be Spam Prevention feature in v3.0? No, It wont make into Stable release but it is on the roadmap for v3.1. It will however not be simple bayesian filtering like our competitors have. Spam is ever evolving and simply tagging certain type of emails as Spam or Non Spam and then comparing keywords will never fix the issue. If it does for you, then you are one of the lucky ones. I receive almost 1500 spam emails daily at varun AT kayako.com, Bayesian filtering was tried and it worked only on 40% of emails. The thing that solved it? Black Lists and "Intelligent" checking by SpamAssasin coupled with Bayesian filtering. The spam is now down to 99.99% which is a big achievement in itself.

2) How will Spam Prevention be done in v3.1? As I said earlier it wont be simple Bayesian filtering but a bit more complex system. A system that carries checks like following on an incoming email:
What is the ratio of Misspelled words to correct spellings?
Do images appear in the HTML Part?
Is there any hidden text in HTML Email?
Does the email contain any links?

A lot of such checks will be combined with Bayesian filtering to form a scoring system akin to SpamAssasin does. This will be a true solution to combating spam, the solutions available right now are more of "gimmicks".

3) So why in v3.1? Why not in v3? This feature requires a lot of complex coding and will delay our v3 release further. We plan to release new builds every month with one major feature once v3 gold goes out.

4) What does Kayako recommend? We will ALWAYS recommend using Black Lists along with SpamAssasin to combat spam in the most efficient manner. People have suggested implementing Black List updating into Kayako but Blacklists are huge and will cause unnecessary overhead in the current product. There are Open Source products out there (SpamAssasin etc) that carry out spam filtering in the most optimal manner the added benefit is that they are tried and tested by millions of users.

Feedback as always is welcome.

Regards,

Varun Shoor

I'm glad to hear from you Varun, it's great to know that even if it's not ready for v3 stable it will be on a close future and won't be something that the competition has, I enjoy owning a software that "Innovates" such as Kayako eSupport is will surely wait for v3.1

I want to thank you for your hard work on v3 It's being appreciated

Would there be a possibility of having a "Spam" folder on the helpdesk for the possible spam e-mails?

[Varun Shoor]

Thanks for the feedback. You can create a Custom Ticket Status called "Spam" under Admin CP > Tickets > Insert Status.

Regards,

Varun Shoor

[TonyBaird]

Something else that bothers me is not just spam but emails coming back and what not. I thought I had this setup right but I still continue to manage to almost crash my server do to the mail getting rejected coming back and being rejected once again. When I thought I had every filter possible still manage to do it sometimes.

Maybe a standard filter for atleast this sort of stuff might be a good idea atleast for v3. I know I'm not alone in knowing what all the filters I should have to prevent this from happening I've read the forums and found others with the same issues.

[Olate]

There is a flood checker in v3 which should stop bouncing problems.

[NickMPN]

Hello All,

I see many of your comments here. I run an online poker network and we get 100s of messages a day. Many of them are spam. Even though I don't personally have to deal with the issues I found that it's a pain for my staff. We run a very interesting system that may help you all.

All our e-mail is directed to a mail server. The mail server is a regular server which checks the validity of the domain, IP etc. It also runs it through some other spam checks that are available on the internet as an add-on service. For us paying the cost to have the service is worth it due to the volume.

But by forwarding our mails to the Kayako support system it allows us to run regular checks on each mail before the system receives it. One thing that would be nice is that Kayako have instead of just one field, have "or"... This would allow me the keep the parser list smaller.

Like I would setup SPAM 1 and set it up like this:

If subject contains:

Wholesale Software
"or"
Software Sales
"or"
Hardware Special

Delete message.

I use TheBat v3 Pro and they have this in their filtering. This has assisted me in not only filter creation but filter duplication as well because each filter is setup to parse a specific type of spam. As Varun said it's changing all the time.

Nick

[Varun Shoor]

Hi Nick,
Give the Email Parser Rules under v3 a whirl and let me know if it is like the way you want them to be

Regards,

Varun Shoor

[ZeroSixty]

I will be requiring user registration to submit tickets (hopefully using the AWBS database ) in order to prevent spam, what happens if an unregistered email address contacts Kayako? Is an admin editable email sent out notifying the sender?

[Varun Shoor]

Not yet, But that option will be available in the next build.

Regards,

Varun Shoor

[humble226]

Originally Posted by Varun [K]

2) How will Spam Prevention be done in v3.1? As I said earlier it wont be simple Bayesian filtering but a bit more complex system. A system that carries checks like following on an incoming email:
What is the ratio of Misspelled words to correct spellings?
Do images appear in the HTML Part?
Is there any hidden text in HTML Email?
Does the email contain any links?

A lot of such checks will be combined with Bayesian filtering to form a scoring system akin to SpamAssasin does. This will be a true solution to combating spam, the solutions available right now are more of "gimmicks".

My suggestion would be to not try and re-invent the wheel. SpamAssassin has already put a massive amount of time into figuring out how to match spam, all you really need are hooks for running tickets through SpamAssassin when they come in, and then configure certain actions (e.g. lower priority, seperate queue, delete ticket, whatever) based on the results (score, etc).

Obviously you can at least reject obvious spam with a couple of procmail recipies right now, but it is probably complicated enough to keep everyone who could benefit from such a system from using it. Also remember that rejecting a valid support request because of a false positive is going to be the surest way to piss off or lose a customer, so integrating SA into a ticketing system is probably going to call for being a little bit more conservative than just rejecting everything spam-like (for example, outright reject everything scored > 15.0, place in a seperate queue or with a lower priority for > 8.0, etc).

[HelpDeskSITE.us]

Originally Posted by Neil-UKWSD

Hi,

The only change between v2 and v3 that will help with spam is the user registration system.

You can set the email queues to require user registration, emails from none registered addresses are disgarded.

For SPAM prevention outside user registration you would need to install something like SPAM Assassin and poll the emails using POP£ or IMAP after having them scanned by SPAM Assassin.

I think that this is the best method. I agree with Neil.

[adrien]

Why do people want Kayako to do this work? Surely it is better done in the mail server that receives the mail? There are also a number of options of POP3 proxies that do spam filtering, so you could download the mail into Kayako through one of those. www.death2spam.net has a really good system.

[sebek]

I agree with you Francisco. We have piping setup and are running spamass and still get hammered with our support@ email address. have thought about using a different email address, except our customers will often email to this address directly.

Would love to see some sort of filter built in.