Sunday, December 27, 2009

Synchronizing two machines

Synchronizing two machines                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                               
  Introduction                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                               
    The method I document here shows how I use unison in combination with                                                                                                                                                                        
    incrontab to keep some folders shared between my work and home machines in                                                                                                                                                                   
    sync almost in real time.                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
  Background                                                                                                                                                                                                                                   
                                                                                                                                                                                                                                                                               
    This is only a rant so you can skip this if you are in a hurry.                                                                                                                                                                              
                                                                                                                                                                                                                                                                               
    Before this setup I had a cron job with a rsync task to sync my shared                                                                                                                                                                       
    folders. The problem is that rsync is one-way sync tool and works well                                                                                                                                                                       
    when you have a master machine where you add/modify/delete files and then                                                                                                                                                                    
    rsync one or more slave machines to the master. In my case I have two                                                                                                                                                                        
    machines that are both master and changes on any of them have to be                                                                                                                                                                          
    propagated to the other.  Of course this master-master configuration can                                                                                                                                                                     
    be done with rsync but you really MUST be very careful on what order you                                                                                                                                                                     
    sync (e.g.  from A to B or from B to A) when you use the --delete switch                                                                                                                                                                     
    (see [1] and [2] fpr details). Not using the --delete switch means that no                                                                                                                                                                   
    file in those machines will ever be deleted unless you manually delete the                                                                                                                                                                   
    file from both sides.  I have been using rsync with a crob job for years                                                                                                                                                                     
    but a few months ago I was victim of the --delete switch. Looking for                                                                                                                                                                        
    alternatives I found unison that does the master-master configuration                                                                                                                                                                        
    amazingly well and at the same time I found about incrontab. Both unison                                                                                                                                                                     
    and incrontab together resulted in a very good setup that I will now                                                                                                                                                                         
    document in the rest of this post.                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                               
  Two way sync of two machines (master-master backup)                                                                                                                                                                                          
                                                                                                                                                                                                                                                                               
    Unison will keep two machines in sync taking care of the --delete flag for                                                                                                                                                                   
    us and even has an option to keep several days of backups of any file it                                                                                                                                                                     
    deletes so you can check once a week if it deleted a file you did not want                                                                                                                                                                   
    deleted.                                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    To install unison in (K)Ubuntu we simply do:                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                               
    ; sudo aptitude install unison                                                                                                                                                                                                                
                                                                                                                                                                                                                                                                               
    Note that unison MUST be installed in the two machines you want to keep in                                                                                                                                                                   
    sync. As additional bonus this tool works in all major operating systems                                                                                                                                                                     
    so you can sync your work and home machines even if you use Windows at                                                                                                                                                                       
    work and Linux at home.                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                               
    Now to sync some folders between machines you issue a simple command:                                                                                                                                                                        
                                                                                                                                                                                                                                                                               
    ; unison /home/user ssh://<ipaddress>//home/user      \                                                                                                                                                                                 
    ; -path .vim -path .vimrc -path .muttrc -path .mutt   \                                                                                                                                                                                       
    ; -path Music -path Photos -path Docs                                                                                                                                                                                                         
                                                                                                                                                                                                                                                                               
    In the above example I sync my vim, mutt configuration files and my Music,                                                                                                                                                                   
    Photos and Docs folders in my home directory. Replace the ipaddress with                                                                                                                                                                     
    the address of the remote machine and note that the double slash (//)                                                                                                                                                                        
    after the ipaddress is not a typo, you MUST have two slashes there when                                                                                                                                                                      
    using absolute paths in the remote side.                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    This command can be executed in any of the two machines and the result                                                                                                                                                                       
    will be the same. Both of them will have the directories selected (-path)                                                                                                                                                                    
    syncrhonized to the last detail.                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
    Since I use ssh as transport I also set up my machines to authenticate                                                                                                                                                                       
    using ssh keys and configure Kwallet [3] as my ssh passphrase manager.                                                                                                                                                                       
    This is very important if we want to automate the task via cron or incron                                                                                                                                                                    
    jobs as I show later in this post.                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                               
    To simplify the synchronization task I created a small bash script that                                                                                                                                                                      
    loads my ssh key manager (keychain), executes unison to synchronize my                                                                                                                                                                       
    machines while logging everything to /tmp/unison.log. 
    <SCRIPT                                                                                                                                                                                                                                 
    < #!/usr/bin/env bash                                                                                                                                                                                                                   
    < LOGGER=/usr/bin/logger                                                                                                                                                                                                                
    < UNISON=/usr/bin/unison                                                                                                                                                                                                                
    < KEYCHAIN=/usr/bin/keychain                                                                                                                                                                                                            
    < $LOGGER -t unison -p cron.info "Unison started for user $HOME"                                                                                                                                                              
    < if [ -e /tmp/unison.lock ]                                                                                                                                                                                                            
    < then                                                                                                                                                                                                                                  
    <   $LOGGER -t unison -p cron.info "Unison already running... abort"                                                                                                                                                          
    < else                                                                                                                                                                                                                                  
    <   echo "running" > /tmp/unison.lock                                                                                                                                                                                      
    <   $KEYCHAIN id_rsa                                                                                                                                                                                                                    
    <   source $HOME/.keychain/$HOSTNAME-sh                                                                                                                                                                                                 
    <   $UNISON $HOME ssh://192.168.1.10/$HOME -batch -log -logfile /tmp/unison.log \                                                                                                                                                       
    <                           -path .vim -path .vimrc -path .muttrc -path .mutt   \                                                                                                                                                       
    <                           -path Music -path Photos -path Docs                                                                                                                                                                         
    <   [ $? -eq 0 ] && $LOGGER -t unison -p cron.info "Unison finished" || $LOGGER -t unison -p cron.info "Unison failed with error $?"                                                                        
    <   rm -f /tmp/unison.lock                                                                                                                                                                                                              
    < fi                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
  Using incrontab for almost real time two way file sync                                                                                                                                                                                       
                                                                                                                                                                                                                                                                               
    Now that I can sync my two machines with a simple script I wanted this to be                                                                                                                                                                 
    automated. The normal way would be using a cron job but a problem I always                                                                                                                                                                   
    had with cron was how to set the time period. Once an hour or a day or a                                                                                                                                                                     
    minute? While researching I found incrontab that is like cron but it reacts                                                                                                                                                                  
    to file system events. With it I could invoke my unison script every time a                                                                                                                                                                  
    file is created or edited that would result in almost real time sync between                                                                                                                                                                 
    my machines.                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                               
    To install incrontab in (K)Ubuntu simply do:                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                               
    ; sudo aptitude install incron                                                                                                                                                                                                                
    ; echo "username" >> /etc/incron.allow                                                                                                                                                                                        
                                                                                                                                                                                                                                                                               
    The first command installs incron and the second one adds your user name                                                                                                                                                                     
    to the list of allowed incron users. If your username is not in the                                                                                                                                                                          
    "incron.allow" file then you wont be able to use incrontab.                                                                                                                                                                        
                                                                                                                                                                                                                                                                               
    A common problem that happens when using rsync or unison from within cron                                                                                                                                                                    
    and incron is how to handle the authentication of the ssh session.                                                                                                                                                                           
    Remember that the cron and incron processes are running in a different                                                                                                                                                                       
    environment as your user so setting a key manager like ssh-agent or                                                                                                                                                                          
    keychain in the current user termial wont work. If you have ever had rsync                                                                                                                                                                   
    work perfectly from a terminal but fail with an error like: unexplained                                                                                                                                                                      
    error (code 255) then you know what I am talking about.                                                                                                                                                                                      
                                                                                                                                                                                                                                                                               
    To solve this problem you first need to set your machines to use                                                                                                                                                                             
    password-less ssh key authentication. This way you can execute remote                                                                                                                                                                        
    commands via ssh without need to input a password.                                                                                                                                                                                           
                                                                                                                                                                                                                                                                               
    Second you need to setup a keymanager like ssh-agent or keychain. I found                                                                                                                                                                    
    that keychain does a better job in keeping a single instance of itself to                                                                                                                                                                    
    handle the keys on multiple processes. So I setup keychain and configure                                                                                                                                                                     
    Kwallet [3] as the interface that asks and stores the ssh pashphrases.                                                                                                                                                                       
    With this setup Kwallet asks me to unlock it at KDE startup and from that                                                                                                                                                                    
    point on it takes care of providing authentication to all processes that                                                                                                                                                                     
    require ssh loging (incrontab included).                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    Once you have setup the key manager you can create an incrontab rule to                                                                                                                                                                      
    invoke the unison script (from the previous section) every time a file is                                                                                                                                                                    
    changed:                                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    ; incrontab -e                                                                                                                                                                                                                                
                                                                                                                                                                                                                                                                               
    This command will open you text editor (vim) with your user's incrontab                                                                                                                                                                      
    configuration file. All you have to do is to input the following line and                                                                                                                                                                    
    save/close the editor:                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                               
    ; /home/user IN_ATTRIB,IN_DONT_FOLLOW,IN_NO_LOOP /home/user/unison.sh                                                                                                                                                                         
                                                                                                                                                                                                                                                                               
    This incrontab rule invokes "unison.sh" every time an attribute changes in                                                                                                                                                         
    a file or folder inside my home directory. The IN_ATTRIB switch may be                                                                                                                                                                       
    overkill and you may prefer to use IN_CREATE or IN_CLOSE_WRITE. For                                                                                                                                                                          
    details on incrontab switches you may read the man page:                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    ; man 5 incrontab 

    This command will open you text editor (vim) with your user's incrontab                                                                                                                                                                      
    configuration file. All you have to do is to input the following line and                                                                                                                                                                    
    save/close the editor:                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                               
    ; /home/user IN_ATTRIB,IN_DONT_FOLLOW,IN_NO_LOOP /home/user/unison.sh                                                                                                                                                                         
                                                                                                                                                                                                                                                                               
    This incrontab rule invokes "unison.sh" every time an attribute changes in                                                                                                                                                         
    a file or folder inside my home directory. The IN_ATTRIB switch may be                                                                                                                                                                       
    overkill and you may prefer to use IN_CREATE or IN_CLOSE_WRITE. For                                                                                                                                                                          
    details on incrontab switches you may read the man page:                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    ; man 5 incrontab                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
    The IN_NO_LOOP avoids the script from being invoked several times when                                                                                                                                                                       
    more than one file changes at a time. Also the script I provided uses a                                                                                                                                                                      
    simple lock mechanism to avoid it from running several times in parallel.                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    As a final note you can open /var/log/syslog and search for all entries                                                                                                                                                                      
    with the unison tag to see if the script is running correctly or failing.                                                                                                                                                                    
    If it is failing you can further chech the /tmp/unison.log file to see why                                                                                                                                                                   
    it is failing. Note that an abort is not an error, it means that then                                                                                                                                                                        
    unison script was invoked several times in succession and we only allow                                                                                                                                                                      
    one instance at any given time.                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                               
  Resources                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    [1] http://www.darcynorman.net/2006/10/03/be-careful-with-rsync-delete/                                                                                    
    [2] http://www.davidgrant.ca/rsync_delete_dangerous                                                                                                                            
    [3] http://piao-tech.blogspot.com/2009/12/manage-ssh-and-gpg-keys-efficiently-in.html                                                        

Thursday, December 17, 2009

Manage SSH and GPG keys efficiently in KDE

Manage SSH and GPG keys efficiently in KDE                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                               
  Introduction                                                                                                                                                                                                                                 
    Here I present a simple way to handle SSH and GPG keys easily in KDE4.                                                                                                                                                                       
    These method presents several advantages:                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
      * Works great with KDE4 that ships with Kubuntu Maverick Meerkat and may                                                                                                                                                                    
        work in other KDE distributions without too much trouble.                                                                                                                                                                                 
      * Use Kwallet as the passphrase manager so you unlock Kwallet once on                                                                                                                                                                       
        login and from there it will handle all passphrase requests.                                                                                                                                                                              
      * Works great with automated tasks (via cron or incron) that use SSH key                                                                                                                                                                    
        authentication (e.g. famous rsync unexplained error 255).                                                                                                                                                                                 
                                                                                                                                                                                                                                                                               
  Pre-requisities                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                               
    Considering that you are reading this post because you need a better way                                                                                                                                                                     
    to manage you ssh and gpg keys then it is safe to assume you have already                                                                                                                                                                    
    generated your ssh/gpg keys and that you know how to use them.                                                                                                                                                                               
                                                                                                                                                                                                                                                                               
  Disable KDE4 from starting ssh-agent and gpp-agent                                                                                                                                                                                           
                                                                                                                                                                                                                                                                               
    Kubuntu 10.10 by default starts the ssh-agent and gpg-agent causing some                                                                                                                                                                     
    conflicts with this setup based on keychain. Using the default                                                                                                                                                                               
    configuration does not seem to use kwallet and certainly does not work                                                                                                                                                                       
    with kmail/mutt so I prefer to disable these and enable keychain instead.                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    To disable the default ssh-agent edit the "/etc/X11/Xsession.options" file                                                                                                                                                         
    and comment out the line that says use-ssh-agent.                                                                                                                                                                                            
                                                                                                                                                                                                                                                 
    To disable the default gpg-agent edit the "~/.gnupg/gpg.conf" file and                                                                                                                                                             
    comment out the line that says use-agent.                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    With these default agent's disabled now we can configure KDE to use                                                                                                                                                                          
    keychain that I consider a superior tool to handle ssh/gpg keys.                                                                                                                                                                             
                                                                                                                                                                                                                                                                               
  SSH/GPG Key management with KWallet                                                                                                                                                                                                          
                                                                                                                                                                                                                                                                               
    First we install the needed software packages:                                                                                                                                                                                               
                                                                                                                                                                                                                                                                               
    ; sudo aptitude install keychain ksshaskpass kwalletcli                                                                                                                                                                                       
                                                                                                                                                                                                                                                                               
    configure the "~/.gnupg/gpg-agent.conf" file so it uses the kwallet                                                                                                                                                                
    pinentry program to manage gpg keys. Simply add the pinentry-program line                                                                                                                                                                    
    or replace it if it already exists with:                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    ; pinentry-program /usr/bin/pinentry-kwallet                                                                                                                                                                                                  
                                                                                                                                                                                                                                                                               
    Now we need to load keychain and all the environment variables it sets                                                                                                                                                                       
    when KDE starts. To do this we simply create a small script, say                                                                                                                                                                             
    "keychain.sh" and put it inside out ".kde/env" directory. The script                                                                                                                                                     
    contains these lines:                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                               
    <SCRIPT                                                                                                                                                                                                                                 
    < #!/bin/sh                                                                                                                                                                                                                             
    < ################################################################################                                                                                                                                                      
    < # Load keychain to handle ssh and gpg keys                                                                                                                                                                                            
    < ################################################################################                                                                                                                                                      
    < if [ -f /usr/bin/keychain ]; then                                                                                                                                                                                                     
    <   if [ -f /usr/bin/ksshaskpass ]; then                                                                                                                                                                                                
    <     export SSH_ASKPASS=/usr/bin/ksshaskpass                                                                                                                                                                                           
    <   else                                                                                                                                                                                                                                
    <     export SSH_ASKPASS=/usr/bin/askpass                                                                                                                                                                                               
    <   fi                                                                                                                                                                                                                                  
    <   /usr/bin/keychain                                                                                                                                                                                                                   
    <   $HOME/.keychain/`hostname`-sh                                                                                                                                                                                                       
    <   $HOME/.keychain/`hostname`-sh-gpg                                                                                                                                                                                                   
    < fi                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    what this does is to setup the SSH_ASKPASS environment variable to use the                                                                                                                                                                   
    ksshaskpass program that handles ssh keys inside KWallet. Then invokes                                                                                                                                                                       
    keychain which starts the ssh-agent and gpg-agent daemons and sets some                                                                                                                                                                      
    environment variables so all KDE applications can see them.                                                                                                                                                                                  
                                                                                                                                                                                                                                                                               
    Finally we must load our ssh/gpg keys into keychain. The best place to do                                                                                                                                                                    
    this is with the KDE Autostart scripts. Simply create a script, say                                                                                                                                                                          
    add_keys.sh, into you ".kde/Autostart" folder that contains something                                                                                                                                                              
    like: 
                                                                                                                                                                                                                                                                               
    <SCRIPT                                                                                                                                                                                                                                 
    < #!/bin/sh                                                                                                                                                                                                                             
    < ################################################################################                                                                                                                                                      
    < # Load keychain to handle ssh and gpg keys                                                                                                                                                                                            
    < ################################################################################                                                                                                                                                      
    < if [ -f /usr/bin/keychain ]; then                                                                                                                                                                                                     
    <   if [ -f /usr/bin/ksshaskpass ]; then                                                                                                                                                                                                
    <     export SSH_ASKPASS=/usr/bin/ksshaskpass                                                                                                                                                                                           
    <   else                                                                                                                                                                                                                                
    <     export SSH_ASKPASS=/usr/bin/askpass                                                                                                                                                                                               
    <   fi                                                                                                                                                                                                                                  
    <   /usr/bin/keychain                                                                                                                                                                                                                   
    <   $HOME/.keychain/`hostname`-sh                                                                                                                                                                                                       
    <   $HOME/.keychain/`hostname`-sh-gpg                                                                                                                                                                                                   
    < fi                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    what this does is to setup the SSH_ASKPASS environment variable to use the                                                                                                                                                                   
    ksshaskpass program that handles ssh keys inside KWallet. Then invokes                                                                                                                                                                       
    keychain which starts the ssh-agent and gpg-agent daemons and sets some                                                                                                                                                                      
    environment variables so all KDE applications can see them.                                                                                                                                                                                  
                                                                                                                                                                                                                                                                               
    Finally we must load our ssh/gpg keys into keychain. The best place to do                                                                                                                                                                    
    this is with the KDE Autostart scripts. Simply create a script, say                                                                                                                                                                          
    add_keys.sh, into you ".kde/Autostart" folder that contains something                                                                                                                                                              
    like:                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                               
    <SCRIPT                                                                                                                                                                                                                                 
    < #!/bin/sh                                                                                                                                                                                                                             
    < ################################################################################                                                                                                                                                      
    < # Load keychain to handle ssh and gpg keys                                                                                                                                                                                            
    < ################################################################################                                                                                                                                                      
    < if [ -f /usr/bin/keychain ]; then                                                                                                                                                                                                     
    <  /usr/bin/keychain id_rsa 0x12345 0x23456                                                                                                                                                                                             
    <  $HOME/.keychain/`hostname`-sh                                                                                                                                                                                                        
    <  $HOME/.keychain/`hostname`-sh-gpg                                                                                                                                                                                                    
    < fi                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                               
    what this Autostart script does is to load your ssh key (id_rsa) and gpg                                                                                                                                                                     
    keys (0x12345, 0x23456) into keychain. The next time you log into a KDE                                                                                                                                                                      
    session this script will ask you if you want to give keychain access to                                                                                                                                                                      
    Kwallet and then ask all the registered key passphrases. Once registered                                                                                                                                                                     
    with kwallet all your applications will be able to use these keys without                                                                                                                                                                    
    asking you for the passphrase each time.                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                               
    Make sure the env and Autostart scripts have exec privileges:                                                                                                                                                                                
                                                                                                                                                                                                                                                                               
    ; chmod +x ~/.kde/env/keychain.sh                                                                                                                                                                                                             
    ; chmod +x ~/.kde/Autostart/add_key.sh