1. Kayako Download customers: we will continue to develop and support Kayako Download beyond July 2017, alongside the new Kayako for existing customers.

    Find out more.

  2. The forum you are viewing relates to Kayako Classic. If you signed up or upgraded to the new Kayako (after the 4th July 2016), the information in this thread may not apply to you. You can visit the forums for the new Kayako here.

kayako 4.70.0 on RHEL with apache 2.4, php 5.4 on AWS ec2 instance

Discussion in 'Installation and setup (Kayako Classic Download)' started by Mike Carifio, Oct 30, 2015.

  1. Mike Carifio

    Mike Carifio Member

    I have installed kayako 4.70.0 on an AWS ec2 "medium" elastic compute node. I've installed "stock" RedHat RHEL 7.1 and then apache2 and php from Amazon's yum repositories. The database is a mysql RDS instance. I've installed xcache, but turned it off for now.

    We have a database of approximately 50 tickets. As we browse tickets, the machine very quickly becomes sluggish and grinds to a halt. We see "too many files open" in the apache error log. It would appear that many open sockets are not closed, specifically:

    $ sudo lsof -u apache|grep sock|grep TCP|wc -l

    will report numbers over 10,000 after about 50 kayako page views.

    I've engaged a kayako consultant, who claims he's never seen this problem. I asked kayako for support. They claim its my fault and there are many successful kayako installations worldwide. I'm not too concerned with the others. I'm quite concerned with my own.

    So. Has anyone seen this problem? How can it be fixed? Can kayako be installed on AWS ec2 instances?
     
  2. Andres Berdugo - Dewak

    Andres Berdugo - Dewak Experienced Member

    Hi Mike,

    We support a couple Kayako instances running perfectly under RHEL, not in AWS ec2 but I think that should be transparent to Kayako.
    I suspect the problem might be related to an Apache configuration, could you paste a bit of the Apache error log?

    I also recommend to switch over to nginx+php fpm+php 5.5+Zend opcache
     
  3. Mike Carifio

    Mike Carifio Member

    I think AWS should be transparent to Kayako also but a confirmation would be useful. Is anyone running a successful installation atop an AWS ec2 instance with an AWS RDS backend?

    As requested, here's a small section of error_log. This goes on and on...

    [Sat Oct 31 09:50:01.424591 2015] [:error] [pid 18798] [client 52.23.31.9:35230] PHP Warning: Unknown: failed to open stream: Too many open files in Unknown on line 0
    [Sat Oct 31 09:50:01.424616 2015] [:error] [pid 18798] [client 52.23.31.9:35230] PHP Fatal error: Unknown: Failed opening required '/var/www/html/cron/index.php' (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0
    [Sat Oct 31 09:50:34.282255 2015] [:error] [pid 18794] [client 76.179.77.80:59500] PHP Warning: Unknown: failed to open stream: Too many open files in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:50:34.282299 2015] [:error] [pid 18794] [client 76.179.77.80:59500] PHP Fatal error: Unknown: Failed opening required '/var/www/html/visitor/index.php' (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:50:54.285109 2015] [:error] [pid 18793] [client 76.179.77.80:59514] PHP Warning: Unknown: failed to open stream: Too many open files in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:50:54.285132 2015] [:error] [pid 18793] [client 76.179.77.80:59514] PHP Fatal error: Unknown: Failed opening required '/var/www/html/visitor/index.php' (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:51:14.320866 2015] [:error] [pid 18794] [client 76.179.77.80:59534] PHP Warning: Unknown: failed to open stream: Too many open files in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:51:14.320896 2015] [:error] [pid 18794] [client 76.179.77.80:59534] PHP Fatal error: Unknown: Failed opening required '/var/www/html/visitor/index.php' (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:51:34.321845 2015] [:error] [pid 18796] [client 76.179.77.80:59554] PHP Warning: Unknown: failed to open stream: Too many open files in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:51:34.321873 2015] [:error] [pid 18796] [client 76.179.77.80:59554] PHP Fatal error: Unknown: Failed opening required '/var/www/html/visitor/index.php' (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:51:54.323974 2015] [:error] [pid 19183] [client 76.179.77.80:59572] PHP Warning: Unknown: failed to open stream: Too many open files in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0
    [Sat Oct 31 09:51:54.324019 2015] [:error] [pid 19183] [client 76.179.77.80:59572] PHP Fatal error: Unknown: Failed opening required '/var/www/html/visitor/index.php' (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0, referer: http://validation.devalidation.com/index.php?/Tickets/Ticket/View/18/false/0

    These errors don't happen initially. The number of open sockets reaches a threshold and then this starts happening.

    I've asked Amazon about this and they hypothesize that mysql is not closing connections. Is there a setting in kayako or php that forces this to be true?
     
  4. Mike Carifio

    Mike Carifio Member

    The AWS hypothesis appears false to me. Here's a snippet of lsof:

    $ lsof -u apache
    ...
    httpd 22461 apache 19u IPv4 733932 0t0 TCP ip-172-31-48-64.ec2.internal:39042->ip-172-31-35-251.ec2.internal:mysql (ESTABLISHED)
    httpd 22461 apache 20u sock 0,6 0t0 732702 protocol: TCP
    httpd 22461 apache 21u sock 0,6 0t0 732726 protocol: TCP

    It's clear that all these open "sock" files are not mysql related.

    $ lsof -u apache |grep ':mysql'|wc -l
    8

    ... which is what I would have expected.
     
  5. JP Serame

    JP Serame New Member

    Hi Mike. Did you find a solution to this problem, re: too many open files? We are currently experiencing this problem and waiting for the Kayako techs to provide light to this dark times.
     
  6. Mike Carifio

    Mike Carifio Member

    No. We redid the whole thing, a new ec2 instance, kayako and reapplied our customizations. I think it had something to do with applying an incompatable php module such as debugging. I could never figure out which one. And this was a major guess. Good luck with this, it's difficult to debug, your machine becomes unstable as you do and Kayako isn't particularly helpful.
     

Share This Page