How To Manage Your Servers With Rex - Best Practice

http://www.howtoforge.com/how-to-manage-your-servers-with-rex-best-practice


(R)?ex is a server orchestration and configuration management tool. With (R)?ex you can manage all your boxes from a central point through the complete process of configuration management and software deployment.
In short words, Rex is like Make. There is a central Rexfile where you can define tasks. These tasks gets executed on remote machines over ssh. The tasks are written in plain perl.
You can get (R)?ex from its website http://rexify.org/.

Preface

In this guide I will use Subversion to manage all the Tasks. You can use any other SCM system as well, as long it supports something similar to Subversion's external directive. I'm using Ubuntu 12.04 but you can use other distributions as well.
I won't append "sudo" on every command, please use "sudo" where it is applicable.
In this guide I will create two example projects. One project is named 'website' and an other called 'database' because in larger companies there is often a split between system administrators and database administrators. Both projects will use the "common tasks" that can be managed for example from a central operations team.
I will use multiple servers:

• Subversion server, svn01
• Database Server, db01
• Web Server, web01
• Workstation, wks01

Set Up Code Repositories

First your have to install all the needed packages. Execute this command on the Subversion server.
Now edit the file /etc/apache2/mods-enabled/dav_svn.conf and paste the following code into it (replacing existing content).

  DAV svn
  SVNParentPath /var/lib/svn
  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /etc/apache2/dav_svn.passwd
  
    Require valid-user
  

Now create the directory /var/lib/svn and all the needed repositories. I will describe the contents of the repositories later.
After we've created the repositories we need to set up the authentication for apache.
Now it is time to restart apache.
Congratulations. Your Subversion Server is now ready. Lets head over to your Workstation and checkout the repositories.

Writing The Tasks

At your workstation you can now checkout the repositories.
First we will add a common task to set up NTP. Here you can add other common tasks later. So change to the Common directory and create a file named NTP.pm.

# Common/NTP.pm
package Common::NTP;
use Rex -base;
task prepare => sub {
   install "ntp";
   file "/etc/ntp.conf",
      source => "files/ntp.conf",
      on_change => sub {
         service ntp => "restart";
      };
};
1;

This creates a task called "prepare". This task gets registered in the "Namespace" NTP. The task will install the package "ntp" if it is not already installed and upload the configuration file to the server. If the content of the file changes it will restart the ntp service. Now, you need to create the ntp.conf file.
Paste this to the file files/ntp.conf. This is a simple default ntp.conf file. Of course you can change this to suite your needs :)

# /etc/ntp.conf, managed with rex
driftfile /var/lib/ntp/ntp.drift
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
server 0.ubuntu.pool.ntp.org
server 1.ubuntu.pool.ntp.org
server 2.ubuntu.pool.ntp.org
server 3.ubuntu.pool.ntp.org
# Use Ubuntu's ntp server as a fallback.
server ntp.ubuntu.com
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery
# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

For the beginning this is enough. So now you can add the new files to the repository.
Now we will add something to the service repository.
Now paste the following code to the Apache.pm Module.

package Service::Apache;
use Rex -base;
task prepare => sub {
   install "apache2";
};
task configure => sub {
   my $param = shift;
   file "/etc/apache2/apache2.conf",
      owner => "root",
      mode => 644,
      content => template("templates/apache2/apache2.conf.tpl", %{ $param });
   file "/etc/apache2/conf.d/security",
      owner => "root",
      mode => 644,
      content => template("templates/apache2/conf.d/security.tpl", %{ $param });
};
1;

And create the templates used by this module.
The contents of templates/apache2/apache2.conf.tpl. I've stripped all comments.

LockFile /var/run/apache2/accept.lock
PidFile /var/run/apache2.pid
Timeout <%= is_defined($::timeout, "300") %>
KeepAlive <%= is_defined($::keepalive, "On") %>
MaxKeepAliveRequests <%= is_defined($::max_keepalive_requests, "100") %>
KeepAliveTimeout <%= is_defined($::keepalive_timeout, "5") %>

    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   0


    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75 
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0


    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75 
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0

User <%= is_defined($::user, "www-data") %>
Group <%= is_defined($::group, "www-data") %>

AccessFileName .htaccess

    Order allow,deny
    Deny from all
    Satisfy all

DefaultType None

HostnameLookups <%= is_defined($::hostname_lookups, "Off") %>
ErrorLog <%= is_defined($::error_log, "/var/log/apache2/error.log") %>
LogLevel <%= is_defined($::log_level, "warn") %>
Include mods-enabled/*.load
Include mods-enabled/*.conf
Include httpd.conf
Include ports.conf
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

Include conf.d/
Include sites-enabled/

The content of templates/apache2/conf.d/security.tpl. I've stripped all comments.

ServerTokens <%= is_defined($::server_tokens, "Prod") %>
ServerSignature <%= is_defined($::server_signature, "Off") %>
TraceEnable <%= is_defined($::trace_enable, "Off") %>

Now we will continue with the MySQL module. Open the file MySQL.pm and add the following content to it.

package Service::MySQL;
use Rex -base;
task prepare => sub {
   install "mysql-server";
};
task configure => sub {
   my $param = shift;
   file "/etc/mysql/my.cnf",
      owner => "root",
      mode => 644,
      content => template("templates/mysql/my.cnf.tpl", %{ $param });
};
1;

And create the file templates/mysql/my.cnf.tpl.

[mysqld]
user     = <%= is_defined($::user, "mysql") %>
pid-file = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port     = <%= is_defined($::port, "3306") %>
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /tmp
lc-messages-dir   = /usr/share/mysql
skip-external-locking
bind-address      = <%= $::eth0_ip %>
key_buffer     = <%= is_defined($::key_buffer, "16M") %>
max_allowed_packet   = <%= is_defined($::max_allowed_packet, "16M") %>
thread_stack      = <%= is_defined($::thread_stack, "192K") %>
thread_cache_size       = <%= is_defined($::thread_cache_size, "8") %>
myisam-recover         = BACKUP
query_cache_limit = <%= is_defined($::query_cache_limit, "1M") %>
query_cache_size        = <%= is_defined($::query_cache_size, "16M") %>
expire_logs_days  = <%= is_defined($::expire_logs_days, "10") %>
max_binlog_size         = <%= is_defined($::max_binlog_size, "100M") %>

[mysqldump]
quick
quote-names
max_allowed_packet   = <%= is_defined($::max_allowed_packet, "16M") %>
[mysql]
[isamchk]
key_buffer     = <%= is_defined($::key_buffer, "16M") %>
!includedir /etc/mysql/conf.d/

Now add all the files to the repository.
Okay, now you have your first common modules. Now it is time to create the tasks for the database project.


Change to the database repository.
We will now add our 2 common repositories as an external repository.
Add the following lines into the editor, save and close it.

lib/Common  http://svn01/svn/common
lib/Service http://svn01/svn/service

After you've set the externals, run an update and commit the new settings.
The svn up command now checks both external repositories for updates. So if anyone adds new services to one of this repositories you will get the new services afert an svn up. Now you can start creating your Rexfile to set up a database server.

set user => "root";
set password => "f00b4r";
include qw/
   Common::NTP
   Service::MySQL
/;
set group => srvdb => "db01";
task "prepare", group => "srvdb", sub {
   
   # run the common ntp task "prepare"
   Common::NTP::prepare();
   # install mysql
   Service::MySQL::prepare();
   # configure mysql
   Service::MySQL::configure({
      key_buffer         => "32M",
      max_allowed_packet => "32M",
   });
   # restart mysql
   service mysql => "restart";
};

First (in line 1 and 2) we set the authentication to use against the target host (db01). Here you can also use key authentication (see http://rexify.org/howtos/start.html or more information). Then we include the Common::NTP and Service::MySQL module so that we can use the tasks from these modules. Every task gets registered as a perl function, too. So we can just call them like any other perl functions. After we've included the wanted modules, we define a new server group called "srvdb" and add the host "db01" to it. It is possible to add multiple hosts to a group. For example:

set group => srvdb => "db01", "db02", "db03";
set group => srvdb => "db[01..03]";

In line 12 we define our first task named "prepare". This task gets executed on all servers in the group "srvdb". This task calls the "prepare" task of our Common::NTP and Service::MySQL module. After the setup is done, we configure the mysql server in line 20 and restart the mysql service in line 26. That's all. This will install, configure and start the MySQL Server. You can now commit everything back to the repository.
To set up your database server you can type the following command:
If you want to list the tasks inside a Rexfile you can use the following command.
After our database is running we can now create the tasks for the website project.

Creating The Website Tasks

First, change to the website project directory.
First we need to define the externals here as well.
Add the following lines into the editor, save and close it.

lib/Common  http://svn01/svn/common
lib/Service http://svn01/svn/service

After you've set the externals, run an update and commit the new settings.
The svn up command now checks both external repositories for updates. So if anyone adds new services to one of this repositories you will get the new services afert an svn up. Now you can start creating your Rexfile to set up a web server.

# Rexfile
set user => "root";
set password => "test";
include qw/
   Common::NTP
   Service::Apache
/;
set group => srvweb => "web01";
task "prepare", group => "srvweb", sub {
   
   # run the common ntp task "prepare"
   Common::NTP::prepare();
   # install apache
   Service::Apache::prepare();
   # configure apache
   Service::Apache::configure({
      timeout => 60,
   });
   # restart apache
   service apache2 => "restart";
};

Save the file and at it to the repository.
Now you're ready to set up your webserver with the following command.
After you've set up your database and your webserver you are ready to deploy an application. For example, if you're getting a ZIP or TAR archive from the development team, you can do this by adding a second task to your Rexfile.

task "deploy", group => "srvweb", sub {
   upload "your-web-application.tar.gz", "/tmp";
   extract "/tmp/your-web-application.tar.gz",
      owner => "www-data",
      group => "www-data",
      to    => "/var/www";
   rm "/tmp/your-web-application.tar.gz";
};

TERIMA KASIH ATAS KUNJUNGAN SAUDARA
Semoga artikel ini bermanfaat bagi saudara. Jika ingin mengutip, baik itu sebagian atau keseluruhan dari isi artikel ini harap menyertakan link dofollow ke http://androidjapane.blogspot.com/2012/05/how-to-manage-your-servers-with-rex.html. Terima kasih sudah singgah membaca artikel ini.