Installing Willie IRC Bot on Debian

Willie is an IRC bot written in Python that I've recently started using. This blog post describes how to install Willie on Debian and as usual I will be using virtualenv to isolate this Python application from the rest of the system.

Installing Python

First you'll need Python.

sudo apt-get install libpython2.7 python2.7 python2.7-dev python2.7-minimal

The following will also be required to enable all the features Willie supports.

sudo apt-get install enchant python2.7-dev libxslt1-dev libxml2-dev

Remove any apt installed Python packages that we are about to replace. The versions of these packages in the Debian repositories soon get stale.

sudo apt-get purge python-setuptools python-virtualenv python-pip python-profiler

Install pip.

wget https://bootstrap.pypa.io/get-pip.py
sudo python2.7 get-pip.py

Use pip to install virtualenv.

sudo pip install virtualenv --upgrade

The Snakepit

Create a "Snakepit" directory for storing all the Python virtual environments.

mkdir ~/Snakepit

Create a virtualenv for Willie

The following will create a new virtualenv called willie using Python 2.7 as the interpreter.

virtualenv -p /usr/bin/python2.7 ~/Snakepit/willie

Working on a virtualenv

Activate the virtualenv for Willie.

source ~/Snakepit/willie/bin/activate

Your shell prompt will change, something like (willie)user@host:~$, while a virtualenv is being worked on to indicate which virtualenv is currently active.

While working on a virtualenv you can pip install what you need or manually install any Python libraries safe in the knowledge you will not upset any other virtualenvs or the global packages in the process. Very useful for developing a new branch which may have different library requirements than the current stable release.

When you are finished working in a virtualenv you can deactivate it by simply executing deactivate.

Install Willie

I've decided to use Python 2.7 to run Willie and therefore have to install backports.ssl_match_hostname which is not required if you use Python 3.3.

pip install willie backports.ssl_match_hostname

Additional functionality

Willie has no external dependencies, besides Python. However, some of the modules do have external dependencies. So install the following Python modules so that I can make use of everything Willie can do.

pip install feedparser pytz lxml praw pyenchant pygeoip ipython --upgrade

Configure Willie

I am not going to explain to how to configure Willie because all that good stuff is very well documented by the project.

But for reference, my default.cfg looks something like this:

[core]
nick = nicofyourbot
user = nicofyourbot
name = Give You Bot A Name
host = chat.freenode.net
use_ssl = true
verify_ssl = true
port = 6697
owner = nicofthebotowner
channels = #example
nickserv_password = ************
prefix = \.
timeout = 120

[db]
userdb_type = sqlite
userdb_file = /home/username/.willie/willie.db

Willie as a daemon

From this point on I assume you've completed the first run configuration of Willie and have .willie/default.cfg in your home directory.

Add the following to /etc/init.d/willie.

#!/bin/sh
### BEGIN INIT INFO
# Provides: willie
# Required-Start: $local_fs $remote_fs
# Required-Stop: $local_fs $remote_fs
# Should-Start: $network
# Should-Stop: $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Willie IRC Bot.
# Description: Start and stops the Willie IRC bot for a given user.
### END INIT INFO

# NOTE! Replace with the user you want to run Willie.
willie_USER="yourusername"

HOMEDIR=$(getent passwd $willie_USER | awk -F: '{print $6}')
DAEMON="$HOMEDIR/Snakepit/willie/bin/willie"
CONFIG="$HOMEDIR/.willie/default.cfg"

startd() {
    if [ -f ${CONFIG} ]; then
        echo "Starting Willie for $willie_USER"
        start-stop-daemon -c $willie_USER -u $willie_USER -x $DAEMON -S -- --config ${CONFIG} --fork --quiet
    else
        echo "Couldn't start Willie for $willie_USER (no $CONFIG found)"
    fi
}

stopd() {
    echo "Stopping Willie for $willie_USER"
    willie_PID=$(pgrep -fu $willie_USER $DAEMON)
    if [ -z "$willie_PID" ]; then
        echo "Willie for USER $willie_USER: not running."
    else
        kill -15 $willie_PID
    fi
}

status() {
    willie_PID=$(pgrep -fu $willie_USER $DAEMON)
    if [ -z "$willie_PID" ]; then
        echo "Willie for USER $willie_USER: not running."
    else
        echo "Willie for USER $willie_USER: running (pid $willie_PID)"
    fi
}

case "$1" in
    start) startd ;;
    stop) stopd ;;
    restart|reload|force-reload) stopd && startd ;;
    status) status ;;
    *) echo "Usage: /etc/init.d/willie {start|stop|reload|force-reload|restart|status}"
       exit 1
       ;;
esac

exit 0

Set the permissions.

sudo chmod +x /etc/init.d/willie

Check that you can start/stop Willie.

sudo /etc/init.d/willie start
sudo /etc/init.d/willie status
sudo /etc/init.d/willie stop

Add willie to the startup/shutdown sequence.

sudo update-rc.d willie defaults

And that's it. Willie is now running as a daemon inside a virtualenv.

ZNC IRC proxy

I have been using the BIP IRC proxy that maintains a persistent connection(s) to a list of IRC channels. However, I've heard good things about ZNC and decided to give it a try.

The purpose of an IRC proxy, or bouncer, is that you can then point your IRC clients to them to maintain a transparent connection from multiple clients and playback the conversations that took place while you were away.

Installing ZNC

The ZNC package for Debian Wheezy are very old, so I decide to install from source.

Install required packages

We first need to make sure we have all the packages required to build ZNC.

sudo apt-get install build-essential libssl-dev libperl-dev pkg-config

Compile ZNC

Now download and compile ZNC.

cd
wget http://znc.in/releases/znc-1.4.tar.gz
tar zxvf znc-1.4.tar.gz
cd znc-1.4
./configure --with-openssl
make
sudo make install

Create a user

Create a separate ZNC user so that ZNC does not need to run as root:

sudo groupadd znc
sudo adduser --system --home /var/lib/znc --group znc

Configuring ZNC

You can use the interactive wizard to configure ZNC which really help create the initial configuration.

sudo -u znc /usr/local/bin/znc --datadir=/var/lib/znc --makeconf

Here is a transcript of how I answered the initial configuration questions.

[ .. ] Checking for list of available modules...
[ >> ] ok
[ ** ] Building new config
[ ** ]
[ ** ] First let's start with some global settings...
[ ** ]
[ ?? ] What port would you like ZNC to listen on? (1025 to 65535): 7778
[ ?? ] Would you like ZNC to listen using SSL? (yes/no) [no]: yes
[ ?? ] Would you like ZNC to listen using both IPv4 and IPv6? (yes/no) [yes]:
[ .. ] Verifying the listener...
[ >> ] ok
[ ** ] Unable to locate pem file: [/var/lib/znc/znc.pem], creating it
[ .. ] Writing Pem file [/var/lib/znc/znc.pem]...
[ >> ] ok
[ ** ]
[ ** ] -- Global Modules --
[ ** ]
[ ** ] +-----------+----------------------------------------------------------+
[ ** ] | Name      | Description                                              |
[ ** ] +-----------+----------------------------------------------------------+
[ ** ] | partyline | Internal channels and queries for users connected to znc |
[ ** ] | webadmin  | Web based administration module                          |
[ ** ] +-----------+----------------------------------------------------------+
[ ** ] And 10 other (uncommon) modules. You can enable those later.
[ ** ]
[ ?? ] Load global module <partyline>? (yes/no) [no]: yes
[ ?? ] Load global module <webadmin>? (yes/no) [no]: yes
[ ** ]
[ ** ] Now we need to set up a user...
[ ** ]
[ ?? ] Username (AlphaNumeric): yournick
[ ?? ] Enter Password:
[ ?? ] Confirm Password:
[ ?? ] Would you like this user to be an admin? (yes/no) [yes]:
[ ?? ] Nick [yournick]:
[ ?? ] Alt Nick [yournick_]:
[ ?? ] Ident [yournick]:
[ ?? ] Real Name [Got ZNC?]: Your Name
[ ?? ] Bind Host (optional):
[ ?? ] Number of lines to buffer per channel [50]: 1024
[ ?? ] Would you like to clear channel buffers after replay? (yes/no) [yes]:
[ ?? ] Default channel modes [+stn]:
[ ** ]
[ ** ] -- User Modules --
[ ** ]
[ ** ] +--------------+------------------------------------------------------------------------------------------+
[ ** ] | Name         | Description                                                                              |
[ ** ] +--------------+------------------------------------------------------------------------------------------+
[ ** ] | chansaver    | Keep config up-to-date when user joins/parts                                             |
[ ** ] | controlpanel | Dynamic configuration through IRC. Allows editing only yourself if you're not ZNC admin. |
[ ** ] | perform      | Keeps a list of commands to be executed when ZNC connects to IRC.                        |
[ ** ] | webadmin     | Web based administration module                                                          |
[ ** ] +--------------+------------------------------------------------------------------------------------------+
[ ** ] And 21 other (uncommon) modules. You can enable those later.
[ ** ]
[ ?? ] Load module <chansaver>? (yes/no) [no]: yes
[ ?? ] Load module <controlpanel>? (yes/no) [no]: tes
[ ?? ] Load module <controlpanel>? (yes/no) [no]: yes
[ ?? ] Load module <perform>? (yes/no) [no]: yes
[ ?? ] Load module <webadmin>? (yes/no) [no]: yes
[ ** ]
[ ?? ] Would you like to set up a network? (yes/no) [no]:
[ ** ]
[ ?? ] Would you like to set up another user? (yes/no) [no]:
[ .. ] Writing config [/var/lib/znc/configs/znc.conf]...
[ >> ] ok
[ ** ]
[ ** ]To connect to this ZNC you need to connect to it as your IRC server
[ ** ]using the port that you supplied.  You have to supply your login info
[ ** ]as the IRC server password like this: user/network:pass.
[ ** ]
[ ** ]Try something like this in your IRC client...
[ ** ]/server <znc_server_ip> +7778 flexiondotorg:<pass>
[ ** ]And this in your browser...
[ ** ]https://<znc_server_ip>:7778/
[ ** ]
[ ?? ] Launch ZNC now? (yes/no) [yes]: no

Running ZNV as a daemon

Here is a /etc/init.d/znc for Debian based on this installation method:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          znc
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: ZNC IRC bouncer
# Description:       ZNC is an IRC bouncer
### END INIT INFO

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin
DESC="ZNC daemon"
NAME=znc
DAEMON=/usr/local/bin/$NAME
DATADIR=/var/lib/znc
DAEMON_ARGS="--datadir=$DATADIR"
PIDDIR=$DATADIR/run
PIDFILE=$PIDDIR/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
USER=znc
GROUP=znc

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
    # Return
    #   0 if daemon has been started
    #   1 if daemon was already running
    #   2 if daemon could not be started
    if [ ! -d $PIDDIR ]
    then
        mkdir $PIDDIR
    fi
    chown $USER:$GROUP $PIDDIR
    start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test --chuid $USER > /dev/null || return 1
    start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --chuid $USER -- $DAEMON_ARGS > /dev/null || return 2
}

#
# Function that stops the daemon/service
#
do_stop()
{
    # Return
    #   0 if daemon has been stopped
    #   1 if daemon was already stopped
    #   2 if daemon could not be stopped
    #   other if a failure occurred
    start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME --chuid $USER
    RETVAL="$?"
    [ "$RETVAL" = 2 ] && return 2
    # Wait for children to finish too if this is a daemon that forks
    # and if the daemon is only ever run from this initscript.
    # If the above conditions are not satisfied then add some other code
    # that waits for the process to drop all resources that could be
    # needed by services started subsequently.  A last resort is to
    # sleep for some time.
    start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON --chuid $USER
    [ "$?" = 2 ] && return 2
    # Many daemons don't delete their pidfiles when they exit.
    rm -f $PIDFILE
    return "$RETVAL"
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
    start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME --chuid $USER
    return 0
}

case "$1" in
  start)
    [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
    do_start
    case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    ;;
  stop)
    [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
    do_stop
    case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    ;;
  status)
    status_of_proc -p $PIDFILE "$DAEMON" "$NAME" && exit 0 || exit $?
    ;;
  reload)
    log_daemon_msg "Reloading $DESC" "$NAME"
    do_reload
    log_end_msg $?
    ;;
  restart)
    log_daemon_msg "Restarting $DESC" "$NAME"
    do_stop
    case "$?" in
      0|1)
        do_start
        case "$?" in
            0) log_end_msg 0 ;;
            1) log_end_msg 1 ;; # Old process is still running
            *) log_end_msg 1 ;; # Failed to start
        esac
        ;;
      *)
        # Failed to stop
        log_end_msg 1
        ;;
    esac
    ;;
  *)
    echo "Usage: $SCRIPTNAME {status|start|stop|reload|restart}" >&2
    exit 3
    ;;
esac

After you've created the script, you must give it the proper permissions to run and add the script to the startup/shutdown sequence.

sudo chmod 755 /etc/init.d/znc
sudo update-rc.d znc defaults

Start the service:

sudo service znc start

Stop the service:

sudo service znc stop

Web configuration

I love that ZNC comes bundled with a web based configuration tool. Just login to https://znc.example.org:7778 to add users, add networks to users and to add channels to networks. Really simple stuff.

IRC client configuration

I use HexChat, but other IRC clients are available. Just add a new Network to HexChat for your ZNC server, use the username, suffixed with the network name you configured in ZNC, and your ZNC password.

Conclusion

I much prefer ZNC to BIP.

  • I really like the web and IRC configuration but I still have the option to configure the config files directly.
  • ZNC is far less cryptic with regard to setting up IRC client connections and user management is much better implemented.
  • When I add a new channel to an existing network in ZNC it automatically appears in my connected clients without the need to restart anything.
  • ZNC's IRC backlogs don't have the confusing double time stamps present in BIP and ZNC is much faster re-establishing connections to my multiple IRC network and channels.
  • Most importantly, ZNC has been far more stable than BIP.

References

Monitorix on Debian

I have a few Debian servers that run at home and on VPSs. I wanted to add some basic systems monitoring to them, but didn't want anything too complicated to look after. I found Monitorix.

Monitorix is a free, open source, lightweight system monitoring tool designed to monitor as many services and system resources as possible. It has been created to be used under production Linux/UNIX servers, but due to its simplicity and small size can be used on embedded devices as well.

Install Monitorix

This install has been tested on Debian Squeeze and Wheezy. First install the dependencies.

sudo apt-get install rrdtool perl libwww-perl libmailtools-perl \
libmime-lite-perl librrds-perl libdbi-perl libxml-simple-perl \
libhttp-server-simple-perl libconfig-general-perl libio-socket-ssl-perl

Now Monitorix itself.

wget -c "http://apt.izzysoft.de/ubuntu/dists/generic/index.php?file=monitorix_3.5.1-izzy1_all.deb" -O monitorix_3.5.1-izzy1_all.deb
sudo dpkg -i monitorix_3.5.1-izzy1_all.deb

At this point Monitorix is installed and running. Point your browser to http://example.org:8080/monitorix/ and enjoy!

Configuring Monitorix

Everything in /etc/monitorix/monitorix.conf is comprehensively documented, just get tweaking.

Each time you update the configuration Monitorix will require a restart.

sudo service monitorix restart

nginx status

If you run nginx then you'll want to drop the following into /etc/nginx/conf.d/status.conf so that Monitorix can monitor nginx.

server {
    listen localhost:80;
    location /nginx_status {
        stub_status on;
        access_log off;
        allow 127.0.0.1;
        deny all;
    }
}
References

subSonic on Debian

Last year I removed all my music from Google Play Music and created my own subSonic server. I really like subSonic but don't use it a huge amount, mostly for syncing some music to my phone prior to going on holiday or business. Therefore, I've made a single one time donation to the project rather than the ongoing monthly usage fee.

Installing subSonic on Debian

This is how I install subSonic on Debian Wheezy.

Install Tomcat.

sudo apt-get install tomcat7

Install subSonic.

apt-get install ffmpeg
sudo mkdir /var/subsonic
sudo chown tomcat7: /var/subsonic
sudo wget -c https://github.com/KHresearch/subsonic/releases/download/v4.9-kang/subsonic.war
sudo cp subsonic.war /var/lib/tomcat7/webapps

Restart Tomcat.

sudo service tomcat7 restart

Login to subSonic by visiting http://server.example.org:8080/subsonic and login with the credentials admin and admin. Make sure you change the password straight away.

Right, that is it. You can stop here and start filling subSonic with your music.

subSonic clients

On the rare occasions that I listen to music via subSonic I use UltraSonic for Android and Clementine on my Arch Linux workstations.

References

Headless cloudprint server on Debian for MFC-7360N

I have a Brother MFC-7360N printer at home and there is also one at work. I wanted to to get Cloudprint working with Android devices rather than use the Android app Brother provide, which is great when it works but deeply frustrating (for my wife) when it doesn't.

What I describe below is how to Cloudprint enable "Classic printers" using Debian Wheezy.

Install CUPS

Install CUPS and the Cloudprint requirements.

sudo apt-get install cups python-cups python-daemon python-pkg-resources

Install the MFC-7360N Drivers

I used the URL below to access the .deb files required.

If you're running a 64-bit Debian, then install ia32-libs first.

sudo apt-get install ia32-libs

Download and install the MFC-7360N drivers.

wget -c http://download.brother.com/welcome/dlf006237/mfc7360nlpr-2.1.0-1.i386.deb
wget -c http://download.brother.com/welcome/dlf006239/cupswrapperMFC7360N-2.0.4-2.i386.deb
sudo dpkg -i --force-all mfc7360nlpr-2.1.0-1.i386.deb
sudo dpkg -i --force-all cupswrapperMFC7360N-2.0.4-2.i386.deb

Configure CUPS

Edit the CUPS configuration file commonly located in /etc/cups/cupsd.conf and make the section that looks like:

# Only listen for connections from the local machine.
Listen localhost:631
Listen /var/run/cups/cups.sock

look like this:

# Listen on all interfaces
Port 631
Listen /var/run/cups/cups.sock

Modify the Apache specific directives to allow connections from everywhere as well. Find the follow section in /etc/cups/cupsd.conf:

<Location />
# Restrict access to the server...
  Order allow,deny
</Location>

# Restrict access to the admin pages...
<Location /admin>
  Order allow,deny
</Location>

# Restrict access to the configuration files...
<Location /admin/conf>
  AuthType Default
  Require user @SYSTEM
  Order allow,deny
</Location>

Add Allow All after each Order allow,deny so it looks like this:

<Location />
# Restrict access to the server...
  Order allow,deny
  Allow All
</Location>

# Restrict access to the admin pages...
<Location /admin>
  Order allow,deny
  Allow All
</Location>

# Restrict access to the configuration files...
<Location /admin/conf>
  AuthType Default
  Require user @SYSTEM
  Order allow,deny
  Allow All
</Location>

Add the MFC-7360N to CUPS.

If your MFC-7360N is connected to your server via USB then you should be all set. Login to the CUPS administration interface on http://yourserver:631 and modify the MFC7360N printer (if one was created when the drivers where installed) then make sure you can print a test page via CUPS before proceeding.

Install Cloudprint and Cloudprint service

wget -c http://davesteele.github.io/cloudprint-service/deb/cloudprint_0.11-5.1_all.deb
wget -c http://davesteele.github.io/cloudprint-service/deb/cloudprint-service_0.11-5.1_all.deb
sudo dpkg -i cloudprint_0.11-5.1_all.deb
sudo dpkg -i cloudprint-service_0.11-5.1_all.deb

Authenticate

Google accounts with 2 step verification enabled need to use an application-specific password.

Authenticate cloudprintd.

sudo service cloudprintd login

You should see something like this.

Accounts with 2 factor authentication require an application-specific password
Google username: [email protected]
Password:
Added Printer MFC7360N

Start the Cloudprint daemon.

sudo service cloudprintd start

If everything is working correctly you should see your printer the following page:

Printing from mobile devices

Android

Add the Google Cloud Print app to Android devices and you'll be able to configure your printer preferences and print from Android..

Chrome and Chromium

When printing from within Google Chrome and Chromium you can now select Cloudprint as the destination and choose your printer.

References
Flattr this