Saturday

MyISAM to InnoDB? Why and How

MySQL has a pluggable storage engine architecture that allows users to select the best storage engine for their application and workload. While InnoDB is by far the most popular storage engine, some users had to rely on MyISAM for its Full text search capabilities, and in some cases better performance in applications not needing transactional capabilities. Those reasons have now vanished with the release of MySQL 5.6. The best MySQL release ever indeed delivers native support for full-text search in InnoDB along with significant performance improvements. Users therefore no longer have to live with trade-offs and can rely on InnoDB for their various use cases.

In this technical webinar Dale Dasker, MySQL Technical Consultant, will review some of InnoDB’s new features, including Full Text search, and improvements that have boosted InnoDB’s performance. Dale will also show you how to migrate your applications and data from MyISAM to InnoDB. He’ll include specific examples and advice for MySQL ISV and OEM customers using MySQL within their products.

Dale will cover the following topics:
  • Portability of InnoDB files (individual .idb files and the Undo log)
  • Performance improvements with InnoDB Read-only transactions
  • Impact of changes to InnoDB page size
  • InnoDB Full Text Search

Sunday

Installing NGINX and PHP-FPM - Setup for NGINX

You've gotten your system ready to install PHP-FPM.
Now this is the most complicated and quite possibly the hardest piece to the setup.
Here is the overwhelming and complicated command set that you will need to enter in order to get NGINX and PHP-FPM installed on your Debian System.
apt-get -t squeeze-backports install nginx-extras; apt-get install php5 php5-fpm php5-common php5-curl php5-dev php5-gd php5-imagick php5-mcrypt php5-memcache php5-mysql php5-pspell php5-snmp php5-sqlite php5-xmlrpc php5-xsl php-pear libssh2-php php5-cli
At this time I create a system user for NGINX.
adduser --system --no-create-home nginx
This command simply creates a system user with no home directory. We will use this User later in our setup.
WOW, that was tough! I know you may be tired, so this is a good time to take a break.

Well now that we have installing NGINX out of the way, we can proceed to setting up NGINX and PHP-FPM to work on your system.
If you are coming from the world of Apache, NGINX will look like Chinese to you. Though you will see that there are some familiar directives and you will see some things that you may recognize. Though I would implore you to go to the NGINX Wiki and read up on how things are done in NGINX. Essentially, if it does not work right it is because you did it wrong. Now that might sound harsh, but it is the truth. NGINX is stupid simple, and one of the things that I had the hardest time with was realizing that there was no need for complicated nonsense in my setup. NGINX allows you to change the outlook on your web application from complication to simplicity. So like Steve Jobs used to always say, "It just Works."

In NGINX everything that you will need to mess with is in the directory /etc/nginx. So for this part of the guide, please go to the directory /etc/nginx and it is here that we will begin changing the configuration files.

There are several default files that you will have to concern yourself with, and then a few others that I created so that I can use in my deployment of Virtual Hosts.
Default Files
  • nginx.conf
  • fastcgi_params
Files that I made to simplify my life
  • security
  • mail.conf
Before we go crazy and start deleting files and changing functions I recommend that we make a backup for the core files. Here is how you can do this simply and fast.
tar -czf ~/NGINX_Config.tar.gz nginx.conf fastcgi_params

Here is the nginx.conf that I have setup for my System. You can modify this to fit the needs of your particular environment, but generally speaking this should be good enough for most production systems.

Please Take Notice: In the nginx.conf file shown below, I have 2 ‘includes’ in the # Virtual Host Configs area. These ‘includes’ are so I can have a separate config file directory and separate virtual host file directory. Now these ‘includes’are not necessary, but make life so much simpler. However if you are set in your old ways and would like to have a very long single configuration file similar to the Apache httpd.conf, NGINX can accommodate that too. Essentially everything would be added to the nginx.conf file. However, I highly recommend you start out with good practices and use the applications resources as they have been designed.
user nginx www-data; worker_processes 4; pid /var/run/nginx.pid; events { worker_connections 768; # multi_accept on; } http { # Basic Settings sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; # Logging Settings log_format gzip '$remote_addr - $remote_user [$time_local] ' '"$request" $status $bytes_sent ' '"$http_referer" "$http_user_agent" "$gzip_ratio"'; access_log /var/log/nginx/access.log gzip buffer=32k; error_log /var/log/nginx/error.log notice; # Gzip Settings gzip on; gzip_disable "msie6"; gzip_vary on; gzip_proxied any; gzip_comp_level 6; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; # Virtual Host Configs include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }

Here are the fastcgi_params
fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx/$nginx_version; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; # PHP only, required if PHP was built with --enable-force-cgi-redirect fastcgi_param REDIRECT_STATUS 200;


Nginx.conf and fastcgi_params are the two basic files that you have to work with in order to setup NGINX. Now here are the custom configs that I added to simplify my life.
First I setup a security file, to fend off some basic annoyances. This is the security file:
## Only requests to our Host are allowed # if ($host !~ ^($server_name)$ ) { # return 444; # } ## Only allow these request methods ## ## Do not accept DELETE, SEARCH and other methods ## if ($request_method !~ ^(GET|HEAD|POST)$ ) { return 444; } ## Deny certain Referers ### if ( $http_referer ~* (babes|forsale|girl|jewelry|love|nudit|organic|poker|porn|sex|teen) ) { return 404; return 403; }

Next I setup a file for the mail directives that are normally in the nginx.conf file. I place these commented out lines in a mail.conf file in the /etc/nginx/conf.d/ folder and I leave the lines all commented out. I do this so that I could use it in the future if I ever choose to, though I am fairly sure that I wont. Essentially this part is optional and not necessary. The file created is mail.conf and it should be placed in the /etc/nginx/conf.d/ directory.
#mail { # # See sample authentication script at: # # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript # # # auth_http localhost/auth.php; # # pop3_capabilities "TOP" "USER"; # # imap_capabilities "IMAP4rev1" "UIDPLUS"; # # server { # listen localhost:110; # protocol pop3; # proxy on; # } # # server { # listen localhost:143; # protocol imap; # proxy on; # } #}


Now that you have the basic NGINX configuration out of the way, it is time to setup some Virtual Hosts. In Debian there are two subdirectories located under the /etc/nginx/ directory. These are sites-available and sites-enabled. The functions for these directories are fairly straightforward. One is the place you house your Virtual Hosts, and the other is where the active Virtual Hosts live. Essentially if you are familiar with Apache, this has the same functionality. You have a Virtual Host Config file in the available directory and you either copy or symlink the files from one place to the other. To symlink a Virtual Host config file from one directory to another, enter this command.
ln -s /etc/nginx/sites-available/THE.VIRTUAL.HOST.FILENAME /etc/nginx/sites-enabled/THE.VIRTUAL.HOST.FILENAME

As you build your Virtual Hosts I recommend that you do this from within the directory /etc/nginx/sites-available.  While following the instructions for this portion of the guide, please navigate to the /etc/nginx/sites-available directory.
Here is the Virtual Host file that I have setup for instances using PHP.
Replace DOMAINNAME with the "DOMAIN NAME" you want to create a Virtual Host for
server { server_name www.DOMAINNAME; rewrite ^(.*) http://DOMAINNAME$1 permanent; } server { listen 80; server_name DOMAINNAME; root /var/www/DOMAINNAME/htdocs; index index.php; include /etc/nginx/security; # Logging -- access_log /var/log/nginx/DOMAINNAME.access.log; error_log /var/log/nginx/DOMAINNAME.error.log notice; # serve static files directly location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt)$ { access_log off; expires max; } location ~ \.php$ { try_files $uri =404; fastcgi_pass unix:/var/run/php5-fpm/DOMAINNAME.socket; fastcgi_index index.php; include /etc/nginx/fastcgi_params; } }
Please review the PHP section of this virtual host template. This is the first reference to using UNIX file sockets for processing PHP.

If you need and or want a Virtual Host that is not using PHP, simply remove the PHP portion of the previous configuration file. That would look like this.
server { server_name www.DOMAINNAME; rewrite ^(.*) http://DOMAINNAME$1 permanent; } server { listen 80; server_name DOMAINNAME; root /var/www/DOMAINNAME/htdocs; index index.php; include /etc/nginx/security; # Logging -- access_log /var/log/nginx/DOMAINNAME.access.log; error_log /var/log/nginx/DOMAINNAME.error.log notice; # serve static files directly location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt)$ { access_log off; expires max; } }
In these virtual host files you will see that there is a line for root /var/www/DOMAINNAME/htdocs. This line is the location where NGINX will look for the files that you are attempting to serve. You should change this so that the Virtual Host File is pointing to the location where you have placed your files.

Once you are finished with the NGINX Aspect of this deployment, meaning you have replaced your files with the ones I have shown here and then created your virtual hosts, you are now ready to configure PHP-FPM.

Here you are going to simply add some POOL information to the PHP-FPM setup that you have already done. Unknowingly when you entered the command to install PHP-FPM, the system setup a default configuration for PHP-FPM. So most of the heavy lifting has been done for you.
You may have to do one thing and that is to create a directory where you want your Sockets to live while they are alive. I placed them here: /var/run/php5-fpm/ You can use this command to create the directory.

mkdir -p /var/run/php5-fpm/
This directory may already exists by simply installing PHP-FPM. This is the place where your file sockets will exist when online. Once you are sure that this directory exists there is nothing further you will need to do with this directory. Sockets will automatically spawn here provided you are using the virtualhost setup and PHP-FPM pool template provided.


To complete the setup move to the directory /etc/php5/fpm/pool.d/. It is here that you will setup the different UNIX Sockets that PHP-FPM will function on. 

Please Note:  In order for your system to function properly you will need to create a new POOL for every Virtual Host that you had setup.
The pool files should follow this naming convention: YOURDOMAIN.conf
Here is a template for the pool files.
[DOMAINNAME] listen = /var/run/php5-fpm/DOMAINNAME.socket listen.backlog = -1 ; Unix user/group of processes user = (THE USERNAME OF THE USER THAT OWNS THE SITE FILES) group = www-data ; Choose how the process manager will control the number of child processes. pm = dynamic pm.max_children = 75 pm.start_servers = 10 pm.min_spare_servers = 5 pm.max_spare_servers = 20 pm.max_requests = 500 ; Pass environment variables env[HOSTNAME] = $HOSTNAME env[PATH] = /usr/local/bin:/usr/bin:/bin env[TMP] = /tmp env[TMPDIR] = /tmp env[TEMP] = /tmp ; host-specific php ini settings here ; php_admin_value[open_basedir] = /var/www/DOMAINNAME/htdocs:/tmp
Please remember to replace all instances of "DOMAINNAME" with then name of your domain.


Now that you have gone through the whole guide and you have installed and created all of the files needed for your installation, you will need to restart NGINX and PHP-FPM. After you restart both services it will be up and working. You will have NGINX running Fast-CGI with PHP-FPM. Your Installation will be running on all separate UNIX Sockets with separate owners per socket ensuring a more secure environment. Furthermore, your NGINX Setup will be fast and furious. In my own speed tests I’ve conducted, my site running on NGINX vs. My Site Running on Apache, my NGINX site was over 3x faster in both response and action. All while on a small 256MB Virtual Server, which in my opinion is ridiculously awesome!
Here is a quick Command to restart Both NGINX and PHP-FPM

service nginx restart; service php5-fpm restart

If there was no error that popped up, you can now go to your domain that you have setup and you will be running good old NGINX and PHP-FPM.
Enjoy!


Another Note..
Just to reiterate, this works and needs nothing else, you can change things to better suit your environment though there is almost nothing else that you need to get the services up and running. Secondly, I have not conveniently left something out of this document. At more than 2500 words, I think I gave too much insight, though I hope this helps.
I would also like to say Thank you to my fellow Rackers, in particular Patrick Cantu, without our bickering about if this could be done, I would have never thought about doing this nor would I have ever done it. I would also like to say thank you to the people over at DotDeb. You people are what make the Open Source Community a great place to develop. You can read more about DotDeb and their repositories here.

curtsy: rackspace

Installing NGINX and PHP-FPM - Setup for NGINX

You've gotten your system ready to install PHP-FPM.
Now this is the most complicated and quite possibly the hardest piece to the setup.
Here is the overwhelming and complicated command set that you will need to enter in order to get NGINX and PHP-FPM installed on your Debian System.
apt-get -t squeeze-backports install nginx-extras; apt-get install php5 php5-fpm php5-common php5-curl php5-dev php5-gd php5-imagick php5-mcrypt php5-memcache php5-mysql php5-pspell php5-snmp php5-sqlite php5-xmlrpc php5-xsl php-pear libssh2-php php5-cli
At this time I create a system user for NGINX.
adduser --system --no-create-home nginx
This command simply creates a system user with no home directory. We will use this User later in our setup.
WOW, that was tough! I know you may be tired, so this is a good time to take a break.

Nginx Support Enables Massive Web Application Scaling

Your web applications need to scale, especially during demanding traffic events. Nginx is a high-performance web server and reverse proxy that can help you do that. Today, we are extending ourManaged Cloud Fanatical Support to include the installation, troubleshooting, patching and performance tuning of Nginx.
Specifics of what is supported can be found in the Knowledge Center article about Cloud Servers with Managed Service Level – Spheres of Support.
At its heart, Nginx is a web server, a very fast web server! The key to Nginx’s speed is its asynchronous or event driven model for serving out objects where each request and response is seen as an individual event that is then forgotten until the next event. By comparison, Apache uses a threaded model where each request ties up resources until the full request is made, responded to and the connection is terminated.
There are three common use-cases where Nginx really stands out, and your Managed Cloud Support Team can help implement for you:
  1. As a reverse proxy / cache in front of Apache
  2. As a reverse proxy /cache in front of an Application Server or Framework
  3. As a replacement for Apache and mod_php

Nginx as a Reverse Proxy / Cache In Front of Apache

Probably the simplest use-case to implement is using Nginx in front of Apache to serve and cache static content while passing dynamic content like PHP back to Apache to serve. Used in this way, you can scale your application right away while changing very little, if any of your code.
Configured this way, you can scale your servers vertically, serving many times more customers by implementing caching and serving static objects directly out of Nginx. If you use any of the most common CMS systems like WordPress, Drupal or Joomla, then you’re probably a good candidate to scale your environment this way.

As a Reverse Proxy /Cache in Front of an Application Server or Framework

Application servers and frameworks like Ruby on Rails, Django, IIS and others handle complex logic and query requests well but generally don’t serve http requests as effectively as Nginx or provide as robust of a security posture as well as a web server.
Nginx can be used in front of those applications to handle the initial requests, cache static content, provide additional security and load balance between multiple application servers allowing you to easily scale out your application.

Nginx as a Replacement for Apache and mod_php

Nginx does not directly replace Apache, so the title of this use-case is a bit misleading. To fully replace Apache with mod_php, an additional application called PHP-FPM is also required to handle the PHP processing. The good news is that it can be easily installed from any of the major distribution repository systems, and our system administrators can help configure PHP-FPM for your environment.
As a replacement for Apache, Nginx can handle all of Apache’s major functions including redirects, authentication, aliases, and server side includes, as well as provides some new capabilities like in-line substitution of page content, streaming video and integration directly with mail servers.

Next Steps

If you’re interested in finding out how Nginx can help you scale your environment, just reach out to your Managed Cloud Support team by ticket, phone or chat and we’ll help you design and implement an Nginx environment customized for your needs.
curtsy: rackspace

Saturday

Step By Step Guide To Install Memcache On Linux


This post is more like a note to me, so that in future I can look up for steps involved in installing memcache on Linux servers like Centos or RHEL.
Those of you who follow me on twitter, will know that me and my friends spends hours trying to install memcache on one of our web server. Normally this is a five minute job, but unfortunately for us those tricks did not work out.
So if you have tried YUM and APT-GET and still could not install the memcache then read on, to find the alternate( read manual, without magic ) way of doing it.
When we talk about Memcache their two things that needs to be installed
  1. Memcache Daemon know as memcached, and
  2. Memcache client for your programing language, in this case PHP.

Installing Memcache Daemon

Note these steps has been taken fromhttp://in2.php.net/manual/en/memcache.installation.php

Steps to install Libevent(memcached dependency)

  1. First we need to check if libevent is installed or not?
    1. type whereis libevent on shell.
  2. If you don’t find it, then we need to download and install it.  Type following sequence of commands on your shell
    1. wget http://www.monkey.org/~provos/libevent-1.4.8-stable.tar.gz
    2. tar xfz libevent-1.4.8-stable.tar.gz
    3. cd libevent-1.4.8-stable
    4. ./configure
    5. make
    6. sudo make install
  3. Finally we need to create a symlink to libevent
    1. type  sudo ln -s /usr/local/lib/libevent-1.4.so.2 /usr/lib on shell.
Now that we have installed the dependencies, we will install the memcahced.

Steps to install Memcached

  1. Download and install, for that type following sequence of commands on your shell
    1. wget http://danga.com/memcached/dist/memcached-1.2.6.tar.gz
    2. tar xfz memcached-1.2.6.tar.gz
    3. cd memcached-1.2.6
    4. ./configure
    5. make
    6. sudo make install
  2. Run Memcached as a daemon (d = daemon, m = memory, u = user, l = IP to listen to, p = port)
    1. type memcached -d -m 1024 -u root -l 127.0.0.1 -p 11211 on shell
Note: versions numbers mentioned here might be outdated, so do check for updated package to download.
Now we have installed and started the Memcache daemon, it’s time to install PHP client.

Installing Memcache Extension for PHP

Some of the steps mentioned below are taken fromhttp://www.sitepoint.com/article/getting-started-with-pear/2/
  1. Let’s determine the PHP version installed and also make sure it is in system path.
    1. type php -v on you shell, it should show something like PHP 5.2.6 (cli) (built: Nov  4 2008 09:25:57)
    2. type whereis php to get the path where PHP is installed in your system.
    3. In your accounts home directory create/edit ‘.profile’ file and add ‘export PATH=$PATH:/usr/local/bin‘ where ‘/usr/local/bin‘ is the folder where php executable is installed.
  2. Install PEAR, type following sequence of commands on your shell
    1. wget http://pear.php.net/go-pear -O go-pear.php
    2. php go-pear.php and follow the installation instructions
    3. ls -l ~/.pearrc
    4. edit the ‘.profile‘ (again) and add exportPATH=/home/yourname/pear/bin:$PATH
  3. Finally Install Memcache
    1. type pecl install memcache on your shell, it will install memcahce extension for PHP.
  4. Edit php.ini to add ‘extension=memcache.so‘, to find the exact php.ini file used by your system
    1. php -i | grep ‘php.ini’
  5. Restart apache, and you are done  type one of the commands below to do that
    1. /etc/init.d/apache  restart , or
    2. service httpd restart
I hope this helps and save some time for you, I have a limited knowledge of Linux, so if some one can improve in the steps, please do so in the comments.

GraphQL with Graph Database

Graph theory is a branch of mathematics that studies graphs, which are mathematical structures that model relationships between objects. A g...