Chamilo - Documentation

Chamilo : Optimization Guide

Documentation > Optimization Guide

In seldom cases, you will need to start looking into efficiency issues with Chamilo. This guide is a work in progress intended to help administrators optimize their Chamilo installation.

Contents

  1. Using xCache, APC or Memcache
  2. Slow queries
  3. Indexes caching
  4. Sessions directories
  5. Users upload directories
  6. Zlib compressed output
  7. Memory considerations for high numbers of users
  8. HTTP caching for media
  9. Avoiding non-fixed values

1. Using xCache or APC

See xCache's website for summary documentation.
Set your xcache.ini configuration (/etc/php5/conf.d/xcache.ini) to match your system. For example, you *could* have something like this (intentionally hiding comments here): 
xcache.shm_scheme =        "mmap"
xcache.size  =                32M
xcache.count =                 2
xcache.slots =                8K
xcache.ttl   =                 0
xcache.gc_interval =           0
xcache.var_size  =           16M
xcache.var_count =            16
xcache.var_slots =            8K
xcache.var_ttl   =            60
xcache.var_maxttl   =        300
xcache.var_gc_interval =     300
xcache.test =                Off
xCache will feel useless until you actually start to put some variables in cache. If you're showing the "Who is online" counter, that's one of the best item there is to implement xCache.
For example, you could implement it this way (in main/inc/lib/banner.lib.php):
$xc = function_exists('xcache_isset');
$number = 0;
if ((api_get_setting('showonline', 'world') == 'true' AND !$user_id) OR (api_get_setting('showonline', 'users') == 'true' AND $user_id) OR (api_get_setting('showonline', 'course') == 'true' AND $user_id AND $course_id)) {
  if ($xc && xcache_isset('campus_chamilo_org_whoisonline_count_simple')) {
    $number = xcache_get('campus_chamilo_org_whoisonline_count_simple');
  } else {
    $number = who_is_online_count(api_get_setting('time_limit_whosonline'));
    xcache_set('campus_chamilo_org_whoisonline_count_simple',$number);
  }
}
$number_online_in_course = 0;
if(!empty($_course['id'])) {
  if ($xc && xcache_isset('campus_chamilo_org_whoisonline_count_simple_'.$_course['id'])) {
    $number_online_in_course = xcache_get('campus_chamilo_org_whoisonline_count_simple_'.$_course['id']);
  } else {
    $number_online_in_course = who_is_online_in_this_course_count(api_get_user_id(), api_get_setting('time_limit_whosonline'), $_course['id']);
    xcache_set('campus_chamilo_org_whoisonline_count_simple_'.$_course['id'],$number_online_in_course);
  }
}
Note that, as xCache is a shared caching system, it is very important to prefix your variables with a domain name or some kind of identifier, otherwise it would end up in disaster if you use a shared server for several portals.
If you use php5-memcache, then this piece of code would look like this (you need to adjust depending on your settings): 
    global $_configuration;
    $_course    = api_get_course_info();
    $course_id  = api_get_course_id();
    $user_id    = api_get_user_id();

    $html = '';
    $xc = method_exists('Memcache','add');
    if ($xc) {
        // Make sure the server is available
        $xm = new Memcache;
        $xm->addServer('localhost', 11211);
        $xc = $xc && ($xm->getServerStatus('localhost',11211)!=0);
        // The following concatenates the name of the database + the id of the
        // access url to make it a unique variable prefix for the variables to
        // be stored
        $xs = $_configuration['main_database'].'_'.$_configuration['access_url'].'_';
    }
    $number = 0;
    if ((api_get_setting('showonline', 'world') == 'true' AND !$user_id) OR (api_get_setting('showonline', 'users') == 'true' AND $user_id) OR (api_get_setting('showonline', 'course') == 'true' AND $user_id AND $course_id)) {
        if ($xc && $xm->get($xs.'wio_count_simple')) {
            $number = $xm->get($xs.'wio_count_simple');
        } else {
            $number = who_is_online_count(api_get_setting('time_limit_whosonline'));
            $xm->set($xs.'wio_count_simple',$number,0,30);
        }

        $number_online_in_course = 0;
        if(!empty($_course['id'])) {
            if ($xc && $xm->get($xs.'wio_count_simple_'.$_course['id'])) {
                $number_online_in_course = $xm->get($xs.'wio_count_simple_'.$_course['id']);
            } else {
                $number_online_in_course = who_is_online_in_this_course_count($user_id, api_get_setting('time_limit_whosonline'), $_course['id']);
                $xm->set($xs.'wio_count_simple_'.$_course['id'],$number_online_in_course,0,30);
            }
        }

An optional additional caching mechanism you may use is the realpath_cache_size and realpath_cache_ttl php.ini parameters. See the PHP documentation for more details.

If you prefer using APC, you can use the same kind of trick as above, just changing the code a little: 
    $xc = function_exists('apc_exists');
    $number = 0;
    if ((api_get_setting('showonline', 'world') == 'true' AND !$user_id) OR (api_get_setting('showonline', 'users') == 'true' AND $user_id) OR (api_get_setting('showonline', 'course') == 'true' AND $user_id AND $course_id)) {
        if ($xc) {
            $apc = apc_cache_info(null,true);
            $apc_end = $apc['start_time']+$apc['ttl'];
            if (apc_exists('my_campus_whoisonline_count_simple') AND (time() < $apc_end) AND apc_fetch('my_campus_whoisonline_count_simple') > 0 ) {
                $number = apc_fetch('my_campus_whoisonline_count_simple');
            } else {
                $number = who_is_online_count(api_get_setting('time_limit_whosonline'));
                apc_clear_cache();
                apc_store('my_campus_whoisonline_count_simple',$number,15);
            }
        } else {
                $number = who_is_online_count(api_get_setting('time_limit_whosonline'));
        }
        $number_online_in_course = 0;
        if (!empty($_course['id'])) {
            if ($xc) {
                $apc = apc_cache_info(null,true);
                $apc_end = $apc['start_time']+$apc['ttl'];
                if (apc_exists('my_campus_whoisonline_count_simple_'.$_course['id']) AND (time() < $apc_end) AND apc_fetch('my_campus_whoisonline_count_simple_'.$_course['id']) > 0) {
                    $number_online_in_course = apc_fetch('my_campus_whoisonline_count_simple_'.$_course['id']);
                } else {
                    $number_online_in_course = who_is_online_in_this_course_count($user_id, api_get_setting('time_limit_whosonline'), $_course['id']);
                    apc_store('my_campus_whoisonline_count_simple_'.$_course['id'],$number_online_in_course,15);
                }
            } else {
                $number_online_in_course = who_is_online_in_this_course_count($user_id, api_get_setting('time_limit_whosonline'), $_course['id']);
            }
        }
     ...

2. Slow queries

Enable slow_queries in /etc/mysqld/my.cnf, restart MySQL then follow using sudo tail -f /var/log/mysql/mysql-slow.log

In Chamilo 1.9 in particular, due to the merge of all databases into one, you might experience performance issue if you have many learning paths with many items in them.
To solve this performance issue, you can execute the following queries manually in your database:
ALTER TABLE lp_item ADD INDEX idx_c_lp_item_cid_lp_id (c_id, lp_id);
ALTER TABLE lp_item_view ADD INDEX  idx_c_lp_item_view_cid_lp_view_id_lp_item_id (c_id, lp_view_id, lp_item_id);
These will be available in Chamilo 1.10 directly, but we cannot put them into Chamilo 1.9 from now on for organizational reasons.

InnoDB Engine and table locking vs row locking

InnoDB is one of the table engines you can use in MySQL. The main advantage of InnoDB is that you can have table locking per row instead of table locking per table. This means that, if one single insert or update query is very slow and executes on a critical table in Chamilo (user, course, etc), it will lock the whole table and no other query will be able to execute, which might seriously affect the efficiency of your database.

Luckily, you can change the engine for one table "on-the-fly", which allows you to effectively check whether this makes a considerable difference. Our recommendation: only do that when seeing that a "SHOW FULL PROCESSLIST" in your database client shows many "Waiting for lock on table [...]".

To change these engines, just launch the following command: 

ALTER TABLE course ENGINE=INNODB;
ALTER TABLE user ENGINE=INNODB;
ALTER TABLE session ENGINE=INNODB;
ALTER TABLE session_rel_course ENGINE=INNODB;
ALTER TABLE session_rel_course_rel_user ENGINE=INNODB;
If used on large tables, this might take a considerable time (can take around 60s for a million rows), so try to execute at night or during lower usage periods.

3. Indexes caching

One good reference: MySQL documentation on multiple key caches

4. Sessions directories

php_admin_value session.save_path 1;/var/www/test.chamilo.org/sessions/

5. Users upload directories

Create 10 directories inside the main/upload/users directory (from 0 to 9) and update your admin settings. This has to be done at install & configuration time, otherwise you might loose user data (or have to write a script for data distribution).

6. Zlib compressed output

Although this will not make your server faster, compressing the pages you are sending to the users will definitely make them feel like your website's responses are a lot faster, and thus increase their well-being when using Chamilo.

Zlib output compression has to be set at two levels: PHP configuration for PHP pages and Apache for images and CSS.

To update the PHP configuration (either in php.ini or in your VirtualHost), use the zlib.output_compression. If you set this inside your Apache's VirtualHost, you should use the following syntax. 
php_value zlib.output_compression 1

Configuring your Apache server to use output compression is a bit trickier. You have to use the mod_deflate module to do it. Your configuration should look like something like this (please read the corresponding documentation before implementing in production).
Easy mode: 
AddOutputFilterByType DEFLATE text/html text/plain text/xml
or, for every content type (dangerous) you can put the following inside a location or directory block:
SetOutputFilter DEFLATE

Advanced mode: 

# Insert filter
SetOutputFilter DEFLATE

# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE masquerades as Netscape, but it is fine
# BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
# the above regex won't work. You can use the following
# workaround to get the desired effect:
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

# Don't compress images
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary

# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
Don't have time or resources to optimize your Chamilo installation yourself? Hire an official Chamilo provider and get it sorted out professionally by specialists. Valid XHTML 1.0 Transitional Valid CSS

Memory considerations for high numbers of users

Some administration scripts *have to* handle lists of all users, and this might have a considerable impact on portals with very high numbers of users. For example, the main/admin/add_users_to_session.php script that handles the registration of users into a specific session, if used with the (non-default) full list of users, will devour about 3KB per user, which, for 100,000 users, translates into the need for around 300MB of RAM just to show this page, and to around 3GB for 1,000,000 users.
This mode is not loaded by default, but could still be selected, leading to a "Fatal error: Allowed memory size ... exhausted" message.
The only non-scripted solution here is to allow for the corresponding amount of RAM for your PHP configuration (memory_limit = 300M) or your specific VirtualHost if you use mod-php5 (php_value memory_limit 300M).

HTTP caching for media

Images are (always) part of the heaviest resources around for a website, together with CSS an JavaScript libraries. Obviously, audio files and videos can be much heavier still, but are much less frequent (for now).

Any web browser will allow you to play a little with HTTP headers. Describing what HTTP really is, is not meant to be in this quick guide, but let's just say that, with the right configuration in your web server, you can "tell" the user's browser to re-use content he already downloaded instead of asking it again from the web server.

It's easy to underestimate the potential gains of doing this. i On a typical web page, your browser will have between 20 (highly optimized) and 200 (not optimized at all) different "items" to download. In Chamilo LMS 1.9, this number is around 40 per page. The first one is the .php file, which tells the browser which other resources to load, then the browser sends requests for these. This includes icons (around 20, depending on the page), JS libraries (around 6), CSS files (around 12) and any other multimedia resource you might have.

Something that is rarely understood by most webmasters or sysadmins is that each one of these resources, unless otherwise configured, will imply a separate request to the server. Even if the server has a mechanism to indicate that a file hasn't changed since the last time, and as such shouldn't be downloaded again, the request is still sent by the browser, received by the server, checked, answered and received again by the browser. While you're waiting for your page to load, your browser is likely to have sent 40 requests and received 40 answers potentially thousands of miles away. You could reduce this to 5, if planned properly.

Even if this last sentence didn't impress you, you'll have to understand that each of these requests comes to your server, uses a server client (or thread, in the best case scenario) which requires memory, reads on disk (unless you've got some nice reverse-proxy mechanism), replies, writes the log for that request and finally goes back to sleep (or to answer the next request). This might all seem invisible for a few requests, but when you start having hundreds of users connecting to your Chamilo site every few seconds, you'll start noticing.

So, how do we make this better?

There is one mechanism that will improve this situation. It's called setting "Expiry" headers, and it is based on the idea that, if we tll the browser how long to keep a specific image in cache, the browser will save it locally, identify it by its URL (so the same image offered on 2 different URLs will be considered as 2 different images) and, when a new page is loaded, if the same image is requested and the expiry date and time have not been reached, it will just load it from cache.
No call to the server, no loading the image, no web server processing, no I/O on the server's disk.
This is probably the single best optimization you can setup on your site.

This measure has to be taken with caution though... some images might be dynamic, or you might want to ensure that the image can only be loaded when the user is connected...
So only a few images (but that's enough by us) will be configurable this way.
And that's alright, because we know exactly which images can be configured this way:

Setting up expiry headers in Apache

The following example is made to be inserted in an Apache's <VirtualHost> block, and should be adapted to fit in other web servers configurations. Some of the settings here might require the activation of the ModExpire and the ModHeader modules: 

  ExpiresActive On
  <Directory "/var/www/chamilo/main/img">
    AllowOverride All
    Order allow,deny
    Allow from all
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
    ExpiresByType application/x-shockwave-flash "access plus 1 month"
  </Directory>
  <Directory "/var/www/chamilo/main/inc/lib">
    AllowOverride All
    Order allow,deny
    Allow from all
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
    ExpiresByType text/css "access plus 1 month"
    ExpiresByType application/javascript "access plus 1 month"
  </Directory>
  <Directory "/var/www/chamilo/main/css">
    AllowOverride All
    Order allow,deny
    Allow from all
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
    ExpiresByType text/css "access plus 1 month"
  </Directory>
  <Directory "/var/www/chamilo/home">
    AllowOverride All
    Order allow,deny
    Allow from all
    ExpiresByType image/gif "access plus 3 hours"
    ExpiresByType image/jpg "access plus 3 hours"
    ExpiresByType image/png "access plus 3 hours"
  </Directory>
These settings explicitly mention the file MIME types that will be put in cache, and for how long. For heavily-optimized sites, where all editors are aware of the settings, understand and apply them, it is common to find cache expiry periods of one full year for images.
If you want to change some content put in cache by the client, though, you will need to change the name of the file (and its reference in the HTML/PHP file that includes it).
This is an easy way to massively reduce your bandwidth (80% reduction?), the speed of page loading for your users and the use of resources on your web server.

Avoiding non-fixed values

Many things in Chamilo are written focusing on the ease of use, even for the administrator. Sometimes, these settings are weighing a little bit more on the system. This is the case, between others, of the mail.conf.php file (being loaded unconditionally) and its CONSTANT "IS_WINDOWS_OS", which is defined by a function call (api_is_windows_os()) at the beginning of main_api.lib.php. The definition of this constant (which is executed at *every* page load) can easily be avoided, and the only place where it is used inconditionally (mail.conf.php) can be modified to set the line as you expect it (depending on whether you use sendmail/exim or smtp). 
$platform_email['SMTP_MAILER']       = 'smtp';
or 
$platform_email['SMTP_MAILER']       = 'mail';
In fact, the complete loading of mail.conf.php can also be avoided if loaded conditionally (with require_once) when sending an e-mail (which is the only case where it is useful).

Authors