Thursday 28 March 2013

Postfix - Mail Solution for RPi

I recently wrote about setting up nullmailer as an ideal solution for mailing out of a Raspberry Pi. It turns out that nullmailer has some severe limitations. Not least of which is that it continues to try to send rejected emails. Forever. Every minute ( by default ).

That's not good. Perhaps there are some configuration tricks for nullmailer that get over this problem. I decided to take a look at how easy it was to setup Postfix as a null (send only) smtp service. It turned out it is relatively simple. Once set up it is very easy to reconfigure Postfix to handle your outgoing mail in any number of ways.

I wondered if first time mail configurers might have trouble getting to grips with what's needed. So I produced a commented script that will do it for you (below).

First, install Postfix and sasl.

sudo apt-get -y update
sudo apt-get -y install libsasl2-modules postfix

You will be asked for a hostname, the type of postfix server to setup and an smtp server during the postfix package install. It does not really matter what you enter as the script will overwrite the configuration anyway but you can accept the defaults for hostname and smtp server while selecting 'Satellite Server' as the type to install.

Alter the 8 configuration variables at the top of the script to suit your requirement and then run with:

sudo perl setuppostfix.txt

As always when running as root ( sudo ) make sure you have read the script and are happy with what it does,

Downloadable settuppostfix.txt here.

If you intend to use your GMail account as an outbound smtp server, you can set up an App Password for authentication.

The script is fully commented but a brief outline of what it does:

given:

  • your hostname is raspberrypi
  • your mailername is raspberrypi.local
  • smtp server smtp.gmail.com on port 587,
  • smtp login name yourname@gmail.com with password 'mypasswd',
  • want to send root and pi mail to mymail@mydomain.com
  • want to use a 'from/replyto' address of raspiadmin@mydomain.com

writes your mailername to /etc/mailname
raspberrypi.local

writes to /etc/aliases
postmaster:    root
root:          mymail@mydomain.com
pi:            mymail@mydomain.com

writes to /etc/postfix/generic
@raspberrypi            raspiadmin@mydomain.com
@raspberrypi.local      raspiadmin@mydomain.com
@localhost.localdomain  raspiadmin@mydomain.com

@localhost              raspiadmin@mydomain.com

writes to /etc/postfix/smtp_auth
[smtp.gmail.com]:587 yourname@gmail.com:mypasswd

writes to /etc/postfix/main.cf using the template at the end of the script.

I hope it proves useful. I'm certainly pleased with the 'upgrade' myself.

-------------------------------------------------------------
#!/usr/bin/perl
use strict;
use warnings;

#----------------------------------------
# THIS SCRIPT WILL OVERWRITE ANY EXISTING
# POSTFIX CONFIGURATION
#----------------------------------------

#----------------------------------------
# This script will perform a simple setup
# of the Postfix mail agent for use as
# a send-only mailer on the Raspberry Pi
# running Raspbian Wheezy/Squeeze/Buster.
# It has been tested as a setup for using
# a Google Mail account for sending email
# and using an ISP smart smtp relay.
# Read through the script to check what it
# does and then run using sudo as a user
# with sudo permissions.
#
# sudo perl setuppostfix.txt
#----------------------------------------

#----------------------------------------
# Configuration
# Alter the 8 items below to suit
# your required configuration. The items
# are described in the following sections
#----------------------------------------

my $fromuser      = 'yourname@gmail.com';
my $adminuser     = 'yourname@gmail.com';
my @adminmaps     = qw( root pi );
my $smtpserver    = 'smtp.gmail.com';
my $smtpport      = 587;
my $loginname     = 'yourname@gmail.com';
my $loginpassword = 'yourloginpassword';
my $mailname      = 'raspberrypi.local';

#----------------------------------------
# $fromuser
#----------------------------------------
# Provide an email address to be used for
# the from and reply-to addresses in emails
# FROM local users on this raspberry Pi.
# When you send mail through your ISP's or
# your own smart smtp relay, there must be
# a valid return address where smtp servers
# between you and the final destination can
# return messages. That ought to be your
# Raspberry Pi but as it is unlikely that
# you have your Pi set up as a visible
# internet server, you must provide an email
# address to be used as the From and
# Reply-To address when local users on your
# Raspberry Pi ( e.g. root@raspberrypi or
# pi@raspberrypi ) send email. The reply
# addresses will be rewritten so that the
# $fromuser email address is used instead.
# This only applies to mail that carries
# a 'local' From address. If your programs
# specifiy a real email address in sent mail
# that won't be rewritten.
# This re-writing is implemented by the
# Postfix  smtp_generic_maps facility.
# If you are using a Google Mail account
# to send your email, it doesn't really
# matter what you put in here. The Google
# servers will re-write whatever you put
# with your Google Mail address.
# If you are using your ISP smtp smarthost
# a genuine email address is probably
# mandatory. 

#----------------------------------------
# $adminuser and @adminmaps
#----------------------------------------
# If you wish, you may send mail destined
# for local users to an external email
# address. That is mail TO local users
# originating on this Raspberry Pi.
# The $adminuser variable contains
# the email address that you want to send
# these mails to. @adminmaps is a space
# separated list of local users whose mail
# you want to forward on.
# This is implemented using the /etc/aliases
# file. For the initial setup this is kept
# to a minimal simple implementation, but
# you can edit the file at any time to
# modify the settings and then run
# sudo newaliases
# to load them into Postfix

#----------------------------------------
# $smtpserver
#----------------------------------------
# This is the the remote smtp relay or
# mail account server. The server for
# Google Mail is already in the script
# so if you are using that you can leave
# it unchanged. Put your smart smtp relay
# in here if you are using one.

#----------------------------------------
# $smtpport
#----------------------------------------
# The port that your smtp server listens
# on. If you are using Google Mail then the
# correct port, 587, is already in the script
# so you can leave that.
# If you are using a smart smtp relay then that
# also most likely listens on port 587.
# You might be using an internal service that
# still listens on port 25, but if you are
# you'll most probably know about that anyway.
#------------------------------------------
# NOTE:
# starttls port 587 Vs wrapped ssl port 465
#------------------------------------------
# Postfix doesn't support 'wrapped ssl'
# communication directly. Servers that require
# this normally listen on Port 465. It is
# possible to use Postfix with wrapped ssl
# but it is beyond the scope of this simple
# script. Google for 'smtp 465 ssl Postfix'.
# Wrapped ssl is quite an old method of
# encrypting the smtp conversation so hopefully
# few people will come across this issue.

#------------------------------------------
# $loginname & $loginpassword
#------------------------------------------
# If you are communicating with a secure
# mail server on port 587 you'll need to
# provide login credentials. For your
# Google Mail account that will be your
# Google Email address and your Google
# login password, OR an app password you
# have created for GMail.
# This will be stored in
# /etc/postfix/smtp_auth and
# /etc/postfix/smtp_auth.db so it is
# essential that these files are readable
# only by root.

#----------------------------------------
# $mailname
#----------------------------------------
# The fully qualified domain name for your
# mail server. This will get written to
# /etc/mailname.
#------------------------------------------
# SCRIPT INNARDS START HERE - You don't
# need to edit anything from here on in
# but there's nothing to stop you if you
# wish.
#------------------------------------------

my $maincf   = '/etc/postfix/main.cf';
my $smtpauth = '/etc/postfix/smtp_auth';
my $generic  = '/etc/postfix/generic';
my $aliases  = '/etc/aliases';

my $servername = $smtpserver;
$servername =~ s/[\[\]]//g;
unless( $smtpport) {
    die 'You must provide an smtp port';
}
$servername = qq([$servername]:$smtpport);

#------------------------------------------
# Make sure we are running via sudo and
# have the necessary permissions.
#------------------------------------------

die 'run this script as root using sudo' if $<;
#------------------------------------------
# Fix mailname and hostname 
#------------------------------------------
my $mailhostname = qx(hostname); chomp($mailhostname); $mailhostname =~ s/[\s\r\n]//g; system(qq(echo "$mailname" > /etc/mailname)) and die 'failed to write /etc/mailname'; #------------------------------------------ # write the main /etc/postfix/main.cf file #------------------------------------------

my $maincftemplate;

{
    my $protocols = 'ipv4';
    my $networks = '127.0.0.0/8';
    
    # check for ipv6
    my $procmods = qx(cat /proc/modules);
    die qq(failed reading /proc/modules : $?) if $?;
    if($procmods =~ /ipv6/) {
        $protocols .= ', ipv6';
        $networks .= ' [::ffff:127.0.0.0]/104 [::1]/128';
    }
    
    $maincftemplate = main_cf_template();
    $maincftemplate =~ s/REPLACEMAILNAME/$mailname/g;
    $maincftemplate =~ s/REPLACEHOSTNAME/$mailhostname/g
    $maincftemplate =~ s/REPLACESERVERNAME/$servername/g;
    $maincftemplate =~ s/REPLACEPROTOCOLS/$protocols/g;
    $maincftemplate =~ s/REPLACENETWORKS/$networks/g;
    
    open my $fh, '>', $maincf
        or die qq(failed to open $maincf : $!);
    print $fh $maincftemplate;
    close( $fh );
    chmod( 0644, $maincf );
}

#-----------------------------------------
# Write the generics file
#-----------------------------------------
if($fromuser){
    open my $fh, '>', $generic
        or die qq(failed to open $generic : $!);
        
    my($host, @discards) = split(/\./, $mailname);
    
    print $fh qq(\@$mailname    $fromuser\n);
    if($host ne $mailname) {
        print $fh qq(\@$host    $fromuser\n);
    }
    print $fh qq(\@localhost.localdomain    $fromuser\n);
    print $fh qq(\@localhost    $fromuser\n);
    close($fh);
    system(qq(postmap $generic))
       and die qq(failed to map generics : $!);
}

#-----------------------------------------
# Write the smtpauth
#-----------------------------------------
if($loginname && $loginpassword) {
    open my $fh, '>', $smtpauth
        or die qq(failed to open $smtpauth : $!);
    
    print $fh qq($servername $loginname:$loginpassword\n);
    close($fh);
    chmod(0600, $smtpauth) or
        die qq(failed to set permissions on $smtpauth : $!);
    
    system(qq(postmap $smtpauth))
       and die qq(failed to map generics : $!);
}

#-----------------------------------------
# Write the aliases
#-----------------------------------------

if( $adminuser && @adminmaps ){
    open my $fh, '>', $aliases
        or die qq(failed to open $aliases : $!);
        
    # add default post address
    print $fh qq(postmaster:    root\n);
    # add maps
    for( @adminmaps ) {
        print $fh qq($_:    $adminuser\n);
    }
    close($fh);
    system('newaliases')
        and die qq(failed calling newaliases : $!);
}

#------------------------------------------
# Restart Postfix and end
#------------------------------------------

system('/etc/init.d/postfix reload')
    and die qq(problems reloading Postfix : $!);
    
    
#------------------------------------------
# Send an email
#------------------------------------------

my $emailtext =
qq(From: root
To: root
Subject: Raspberry Mail Configuration
Content-type: text/plain

Your Raspberry Postfix is configured on $mailname
with the following configuration:

/etc/postfix/main.cf <<
$maincftemplate
);

{
    open my $fh, '|/usr/sbin/sendmail -t'
        or die 'could not open pipe to sendmail';
    print $fh $emailtext;
    close($fh);
}

print qq(Postfix configuration complete\n);

#-----------------------------------------
# Templates
#-----------------------------------------

sub main_cf_template {
    my $template = <<'MAINCFTEMPLATE'
# Basic Null ( send only ) Postfix

smtpd_banner = Raspberry Pi Mail
biff = no
append_dot_mydomain = no
readme_directory = no
myhostname = REPLACEMAILNAME
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = REPLACEHOSTNAME, REPLACEMAILNAME, localhost.localdomain, localhost
relayhost = REPLACESERVERNAME
inet_protocols = REPLACEPROTOCOLS
mynetworks = REPLACENETWORKS
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = loopback-only
smtp_generic_maps = hash:/etc/postfix/generic

# TLS

smtp_use_tls=yes
smtp_sasl_auth_enable=yes
smtp_sasl_password_maps=hash:/etc/postfix/smtp_auth
tls_random_source=dev:/dev/urandom
smtp_sasl_security_options=noanonymous
smtp_tls_security_level=may

MAINCFTEMPLATE
;
    return $template;
}

1;

Tuesday 26 March 2013

Sharing Directories with Windows, Linux and Mac OS X

Yesterday I posted about setting up Netatalk so I could see some directories on my Raspberry Pi from a Mac OS X machine.

To recap, my goal was to be able to use a remote debugger and IDE on the Mac to develop code running on the Raspberry Pi. The IDE I use is Komodo.  It is proprietary application but I can't bring myself to relinquish its features.

Whilst Komodo can remotely debug quite happily without any integration with the remote system, to take advantage of many of its features  and persist your settings across invocations of the remote code it needs access to the code source files. For the Mac, the netatalk setup gave me that.  Next I wanted to be able to access the same Raspberry filesystem from Windows and Linux.

Samba is an implementation of CIFS (Common Internet File System), a network sharing protocol commonly used by MS Windows operating systems to share files and resources. We can use it to share some directories on our Raspberry Pi so that they look like Windows shares to the other machines on our network. The advantage of Samba is that in addition to our Windows machines, the Mac OS X and Linux machines on our network will be able to access the shares without difficulty. We can provide for all three client types with a single service running on the Raspberry Pi.

Installing Samba on the Raspberry Pi

On the Raspberry Pi we install the Samba packages.

sudo apt-get update
sudo apt-get install samba samba-common-bin

The configuration file for Samba is at /etc/samba/smb.conf. We won't edit this directly. Firstly, we want to ensure we keep a copy of the original. Assuming you are running as user pi on the Raspberry Pi:

cp /etc/samba/smb.conf /home/pi/smb.conf.orig

Samba provides a utility that will check your configuration file, and output the result stripped of comments and default settings. We will use that to produce our configuration file from a master that we shall edit.

Create the master file for editing:

nano /home/pi/sambamaster.conf

The following is my master config file that you may paste and edit. You will need to edit it.

[global]
# Change to your workgroup name
  workgroup    = WORKGROUP

# Change to your chosen unique name
  netbios name = RASPMODELB

  server string = %h server
  map to guest = Bad User
  obey pam restrictions = Yes
  pam password change = Yes
  passwd program = /usr/bin/passwd %u

# The 'passwd' chat entry should be on a single line

  passwd chat = *Enter\snew\s*\spassword:* %n\n
        *Retype\snew\s*\spassword:* %n\n
        *password\supdated\ssuccessfully* .

    unix password sync = Yes
  syslog = 0
  log file = /var/log/samba/log.%m
  max log size = 1000
  dns proxy = No
  usershare allow guests = Yes
  panic action = /usr/share/samba/panic-action %d
  idmap config * : backend = tdb
    server min protocol = SMB2

[piraspmodelb]
    comment = Pi Home Directory
    path = /home/pi
  valid users = pi
  read only = No
  create mask = 0644
  browseable = Yes

[raspberryusbdrive]
  comment = Raspberry Pi USB Drive
  path = /media/hdd/pi
  valid users = pi
  read only = No
  create mask = 0644
    browseable = Yes

This setup is good for any network where you are not running a Windows Domain Controller. I have shared two folders, the user pi home directory and the 'pi' subdirectory of a usb hard drive I have connected to this Raspberry Pi. The shares will appear as 'piraspmodelb' and 'raspberryusbdrive'.  Obviously you may only want to share the 'pi' user home directory so you will only have one share section.

Samba offers the facility to automatically share home directories to logged in users under the config setting '[homes]' in the conf file. I did not want this as the share will be called after the user name (in my setup 'pi') and some clients (Mac OS X) will auto-mount this under the share name (/Volumes/pi). If I have a second Raspberry Pi and I use the same setup, then the home directory for user 'pi' on  that Raspberry Pi will be mounted as /Volumes/pi-1. The problem with this is that the assignment of the names is not stored between invocations. So the next time I reboot my Mac /Volumes/pi and /Volumes/pi-1 may point to different things depending on which one I access first.

If you would prefer to use the standard Samba [homes] share for user home directories you can use this link to a simple smb configuration example.

I chose the name 'piraspmodelb' as I'm currently the lucky owner of two Raspberries, a Model A and a Model B so I'm easily aware of whose home directory on which Pi I'm pointing at.

Note that I made both shares 'browseable'. This simply means they will 'appear' to network browsers that have not logged in as a particular user yet but has no effect on whether the files in the share can be viewed. You may need at least one share to be 'browseable' as some client implementations fall over without this.

After you have saved the file and exited 'nano' you can test the syntax and overwrite the actual configuration at the same time.

sudo sh -c 'testparm -s /home/pi/sambamaster.conf  >/etc/samba/smb.conf'

You might have expected to be able to do:
    testparm -s /home/pi/sambamaster.conf  >/etc/samba/smb.conf
but you cannot. The command redirection  '>' would be applied by your current user shell to the output of 'sudo' and you don't have permissions to write to /etc/samba/smb.conf. We have to call the 'sh' shell command with sudo and pass the whole command we want to run wrapped in quotes as the -c parameter.

As part of its integration with Windows networks, Samba can synchronise changes in the Windows access passwords with the unix passwords of users on the Raspberry Pi. We probably don't care too much about that for this setup, but we do nevertheless have to setup our Samba password. Assuming you are using user 'pi':

sudo smbpasswd -a pi

You will be prompted for your pi user password, which if you have not changed from the default will still be 'raspberry'.

When that's complete we can restart the Samba services on the Raspberry Pi:

sudo service smbd restart

Our shares should be visible to client machines on our network.

Connecting to Shares from Mac OS X

Open Finder and from the menu choose

Go -> Connect to Server

In my case I gave the Raspberry a netbios name of 'raspmodelb' so the server I want to connect to is

smb://raspmodelb


When connecting your user and password will be requested which in my case remains 'pi' and 'raspberry'. I select both the available shares to mount ( in my case 'piraspmodelb' and 'raspberryusbdrive' as defined in my smb.conf). The shares are now both browseable in Finder under 'raspmodelb'.



On the filesystem the shares are mounted as:

/Volumes/praspmodelb
/Volumes/raspberryusbdrive

Connecting to Shares from Windows

In Explorer type the netbios name of your Raspberry ( \\raspmodelb for my example ) in the address bar and press enter. This should display your shares.






It is my preference to map network drives for each of the shares. Right click on the share name and choose 'Map Network Drive.'. Remember to select 'Connect using a different user name' in the resulting dialog and enter your pi username and password.

Connecting to Shares from Linux

For Linux we'll mount the shares as part of our file system. My tests for this were done using Ubuntu 12.04 which seemed to have everything necessary installed by default. On other systems, the commands will be the same but you may have to install packages to support CIFS.

For the shares we created on the Raspberry Pi we'll need two mount points. In my case I simply named the mount points after the share names, but this isn't a requirement. You may choose whatever names you wish.

sudo mkdir /media/piraspmodelb
sudo mkdir /media/raspberryusbdrive

Then we mount the drives. I have found that this works best without problems if the Raspberry Pi is referenced by IP address. Get the IP address of your Raspberry using the command 'ifconfig' on the raspberry. Unless you have some custom setup you want the IP address of eth0. For this example we'll use an address of 192.168.0.5 for the Raspberry Pi.

On your Linux client:

sudo mount -t cifs //192.168.0.5/piraspmodelb   \
        /media/piraspmodelb -o user=pi,password=raspberry


sudo mount -t cifs //192.168.0.5/raspberryusbdrive  \
        /media/raspberryusbdrive -o user=pi,password=raspberry


This may be all that you will need to get a working setup but it relies on a coincidental match between the numeric uid of your user on the Linux client and the numeric uid of the 'pi' user on your Raspberry.

On the Raspbian Wheezy, the user 'pi' has the numeric uid 1000 and group gid 1000. For many users with their own personal installations of a Linux distribution the user they created when first installing the distribution and continue to use as a general login also has a uid of 1000 and a gid of 1000. So when the CIFS client is mapping the ids as a user with uid and gid 1000 exists on both systems, all works well.

If however your login user on your Linux Client doesn't have the uid and gid 1000, but some user on your system does, then all the files in the 'pi' home directory on your Raspberry that you mount on the Linux Client will appear to belong to user 1000 on your Linux Client - which is not your current login user. You will be restricted according to the permissions on each file and directory but as they likely will all belong to user 1000, you won't have write access to anything.

You can overcome this issue by specifying a numeric uid and gid when mounting the Samba share. If the login user you want to have read / write access to the mounted files has a uid of 1002 and a gid of 1003 then you would mount with:


sudo mount -t cifs //192.168.0.5/piraspmodelb   \
    /media/piraspmodelb                         \
    -o user=pi,password=raspberry,uid=1002,gid=1003


and all should work OK.

It is possible to alter your /etc/fstab on the Linux client so that the Raspberry Pi shares are mounted automatically. I don't do this because I don't like putting entries in fstab for devices that are often not present - and my Raspberry Pi is often powered off while I mess with circuitry.

I don't type the above out everytime either though.  A couple of bash scripts do the job for me:

mountraspberry.sh
#!/bin/sh

mount -t cifs //192.168.0.5/piraspmodelb \
    /media/piraspmodelb \
    -o user=pi,password=$1



mount -t cifs //192.168.0.5/raspberryusbdrive \
    /media/raspberryusbdrive \
    -o user=pi,password=$1


umountraspberry.sh
#!/bin/sh
umount -l /media/piraspmodelb
umount -l /media/raspberryusbdrive

called as

sudo ./mountraspberry.sh raspberry

sudo ./umountraspberry.sh

Monday 25 March 2013

Connecting to Raspberry Pi with Mac OS X

I've been using a Windows machine as my main desktop for any number of years but last Friday the current incarnation of 'homemachine' accompanied one of its occasional blue screens of death with a blue pall of smoke.

So I'm trying out my Mac Mini as a regular desktop using only 'free' stuff. I've become a bit lazy over the years using a gui for everything but I'm getting back into command line operation quite quickly for many things.

One thing I have to write about is just how easy it is to provide yourself with visibility of your Raspberry Pi file system in Finder.

The following works on the current Raspbian Wheezy.

install netatalk (which will install avahi as a dependency)

sudo apt-get update
sudo apt-get install netatalk

create and open a config file for afp

sudo nano /etc/avahi/services/afpd.service

paste / type the following into the file.


<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
   <name replace-wildcards="yes">%h</name>
   <service>
      <type>_afpovertcp._tcp</type>
      <port>548</port>
   </service>
</service-group>

Save the file and restart the avahi service

sudo /etc/init.d/avahi-daemon restart

That's it. Now in your Mac, open Finder and open the network. Your Raspberry Pi will be visible. Open it. An initial connection attempt will fail. Choose 'Connect As' and enter a username / password ( pi / raspberry ). You will see your home directory listed as "Home Directory".

Further options, if you need them. can be configured on your Pi in /etc/netatalk/AppleVolumes.default

Information on the options for Netatalk here.

I came across the base information on how to do this at Getting Started with Raspberry Pi  & MacOSX

Sunday 24 March 2013

Nullmailer on Raspberry Pi

Edit: 2013-03-28 - Nullmailer Problems

  • There are some problems with nullmailer that caused me to change to Postfix. I've written about setting up PostFix here.

    Nullmailer continues to attempt to send even permanently rejected mails every time the queue handler runs. I suppose it would be possible to write a script to delete queued items over a certain age and synchronise this with Nullmailer. But given the limitations that Nullmailer's minimalist implementation imposes, I switched to using Postfix with a null (send only) setup.


At some point you are probably going to want to send email from your Raspberry Pi. If for nothing else, it would be nice to receive the system notifications that processes often mail to 'root' or the login user.

My own need to setup a mail system has arisen because I want to schedule jobs using cron as part of a project.

By default, if a job scheduled using cron fails, its output is mailed to the running user (normally root).  I want this email to be delivered to an external user, rather than sitting in the root mailbox on my Pi.

I find my favourite tool for this is a simple mail transport agent called nullmailer. I've used the tool in the past on other systems but thought I'd check Google for any Raspberry Pi specific advice. I was surprised to find setup and configuration descriptions were sparse so I put the following together using a smattering of experience and the application help produced by the following:

man nullmailer-queue
man nullmailer-inject
man nullmailer-send
/usr/lib/nullmailer/smtp --help
/usr/lib/nullmailer/qmqp --help

Installation

You can install nullmailer on Raspbian Wheezy with the usual:

sudo apt-get update
sudo apt-get install nullmailer

During installation you will be asked for your required hostname and a remote mailer. You can just accept the (probably non-working) defaults for now.

Once nullmailer is installed, we can configure it. Configuration is achieved by amending or creating several configuration files. All but one of the nullmailer configuration files (/etc/nullmailer/remotes) contains a single line with one entry.

Quick Setup Using a Google Mail Account

Edit the file /etc/nullmailer/remotes using nano

sudo nano /etc/nullmailer/remotes

It should contain a single line containing the following parameters (replaced with your Google account and password, of course). The examples cover two or more lines due to restrictions of the web page format. You must put all the arguments on a single line in your configuration file. 

smtp.gmail.com smtp --port=465 --auth-login --user=you@gmail.com 
                    --pass=password --ssl

Optionally, if you want mail for local users ('root', 'pi' etc) to be forwarded to an external address, edit /etc/nullmailer/adminaddr so that it contains a single line with your forwarding email address in it.

Finally, by default nullmailer will retry any failed messages once every minute. This is undesirable. We only want to try and send a message once.

sudo nano /etc/nullmailer/pausetime

Enter the number 0 in this file to put nullmailer in one shot mode and save the file. It will still try to send all outstanding messages every time the system adds a new one to the queue, but this is OK.

Restart nullmailer

sudo /etc/init.d/nullmailer restart

You are ready to go. Test that  you can send email with:

echo "My Test Message" | sendmail youremail@yourdomain.com

Detailed Configuration

After a default installation on Raspbian Wheezy, the following configuration files will have been created:
  • /etc/mailname
  • /etc/nullmailer/adminaddr
  • /etc/nullmailer/defaultdomain
  • /etc/nullmailer/remotes
Nullmailer also uses the following optional configuration files that you may create and edit:
  • /etc/nullmailer/me
  • /etc/nullmailer/defaulthost
  • /etc/nullmailer/helohost
  • /etc/nullmailer/idhost
  • /etc/nullmailer/pausetime
  • /etc/nullmailer/sendtimeout
After you have edited any configuration files, always restart nullmailer with

sudo /etc/init.d/nullmailer restart

/etc/nullmailer/me

This file is used on systems that don't have /etc/mailname present. Your Rasbian Wheezy has /etc/mailname present so you can ignore this config file. Delete /etc/nullmailer/me if you have created one during a previous configuration of nullmailer.

/etc/mailname

This is normally the same as the output from /bin/hostname. Some programs may use this file to create the domain part of an email address. You can leave this at the system default if you are sending out mail from an email account such as your Google mail account.

If you are using a remote smtp smarthost and you have an entry in /etc/nullmailer/adminaddr you may have to reconfigure the hostname in this file. Programmes such as the 'cron' scheduler will use the value in /etc/mailname to form their 'from' address. Nullmailer does not rewrite full domain addresses where they exist. If your /etc/mailname contains a host name that does not really exist, such as 'raspberrypi.somedomain', then this is used as the from address (and therefore the default reply-to address). The smtp server you are connecting to may reject your mail if it cannot establish a viable return address.

If you are using an external smart smtp relay then your only option is probably to put your real domain name that accepts your email here. For example, for me that would be 'znix.com'.  When you read through the descriptions of the other configuration files below remember that for my case because I have 'znix.com' in /etc/mailname, programs such as 'cron' will send mail To root@znix.com, From root@znix.com. Nullmailer will regard the To address as being for a local user ( the domain is in /etc/mailname ) and so will send it to the email address in /etc/adminaddr. The smtp relayer will see 'znix.com' as a valid return mail host so will accept the mail.

With an internal smart smtp host you can set up that host to rewrite the from and reply-to addresses for mail coming from your internal network.

/etc/remotes

This file contains a list of remote email servers that you wish to send your outgoing mail to. When mail is ready to be sent, nullmailer will try each of the configured servers in turn until it succeeds in sending the mail. If all servers fail, nullmailer will leave the messages in a queue and attempt the process again after the period specified in /etc/nullmailer/pausetime.

Note: Any of the configuration entries below must appear in /etc/nullmailer/remotes as a single line for each server entry.

The basic format for a server entry in /etc/nullmailer/remotes is:
mail.domain.com smtp

however, you are unlikely to have a mail server that accepts direct unauthenticated connections on port 25 so nullmailer allows you to pass several parameters to control how you connect to the server.

server requires plain authentication (unusual)
mail.domain.com smtp --port=25 --user=you --pass=passwd

server requires login authentication
mail.domain.com smtp --port=587 --auth-login --user=you@isp.com 
                     --pass=passwd

server supports TLS login authentication
mail.domain.com smtp --port=587 --auth-login --user=you@isp.com 
                     --pass=passwd --starttls

server listens on SSL port
mail.domain.com smtp --port=465 --auth-login --user=you@isp.com
                     --pass=passwd --ssl

Some ISP's set up their mail servers using SSL certificates with common names like *.domain.com. If nullmailer is connecting to the server mail.domain.com then it will regard any certificate with a common name other than mail.domain.com as insecure (correctly) and will not send your login details. You can pass an option so that nullmailer accepts any SSL certificate. In this case you are only using the certificate to encrypt your login details and not relying on it to identify the remote server.

accept unverified ssl certificate
mail.domain.com smtp --port=465 --auth-login --user=you@isp.com 
                     --pass=passwd --ssl --insecure

The nullmailer version in Raspbian Wheezy can also handle authentication using client SSL certificates over SSL / TLS.  The following options can be added to your server line in /etc/nullmailer/remotes


--x509certfile=<client certificate file>
--x509cafile=<path to certificate authority ca file>
               (default /etc/ssl/certs/ca-certificates.crt)
--x509crlfile=<certificate revocation list (crl)>
--x509fmtder   nullmailer expects files in PEM format. use 
               this flag if x509 files are in DER format.        

Nullmailer also supports the QMQP protocol. Just replace 'smtp' with 'qmqp' in your /etc/nullmailer/remotes configuration line(s).


Google Mail Remotes Recipe

either of the following will work in the remotes file to use a Google mail account to send email.
smtp.gmail.com smtp --port=587 --auth-login --user=you@gmail.com 
                    --pass=password --starttls

smtp.gmail.com smtp --port=465 --auth-login --user=you@gmail.com 
                    --pass=password --ssl

/etc/nullmailer/adminaddr

Processes on your Raspberry Pi that create emails will send them to the local user running the process. The email address will normally be created by prepending the login name to either 'localhost', the output of /bin/hostname or the content of /etc/mailname.

For example;
    root@localhost
    pi@raspberrypi.local

If you put an email address in /etc/nullmailer/adminaddr, then all email addressed to any user at the domains 'localhost' or the domain contained in /etc/mailname will be sent to the email address you enter. For example, if /etc/nullmailer/adminaddr contains

myname@gmail.com

Then mail for root@localhost will be sent to myname@gmail.com. If the content of /etc/mailname is raspberrypi then any mail address to, for example, pi@raspberrypi will be sent to the address in /etc/nullmailer/adminaddr.

/etc/nullmailer/defaulthost

If you are using an email account such as your Google mail account to send mail, then this file is not essential. It may be useful if you use your ISP's smtp server or if you are using a local smtp server.

If this file contains a hostname, then any email address that does not contain any hostname will have this host appended. This applies to all addresses ( From, To, Cc, etc). If nullmailer receives an email addressed to 'root' from 'pi' and /etc/nullmailer/defaulthost contains 'myraspberry.mydomain.net' then the addresses will be changed to 'root@myraspberry.mydomain.net' and 'pi@myraspberry.domain.net'.

If the file /etc/nullmailer/defaulthost is empty, then the domain in /etc/mailname is used.

On my own system I have a local smtp host that then relays mail to my ISP's smtp relay. The ISP smtp relay is fussy about the reply-address in the mail being a resolvable host. I set my /etc/nullmailer/defaulthost file to contain znix.com so the mail from my raspberries appears to come from root@znix.com or pi@znix.com. On the receiving end I have filters set up to reject mail addressed to 'root' unless it originates from one of the raspberries.

/etc/nullmailer/defaultdomain

If you are using an email account such as your Google mail account to send mail, then this file is not essential. It may be useful if you use your ISP's smtp server or if you are using a local smtp server.

If this file contains a domain name, then any email address that does not contain a hostname that includes a full stop ( period ) other than 'localhost' will have this domain appended.  This includes any hostname taken from /etc/nullmailer/defaulthost or /etc/nullmailer/idhost.

This applies to all addresses ( From, To, Cc, etc). If nullmailer receives an email addressed to 'root@raspberry' from 'pi@localhost' and /etc/nullmailer/defaultdomain contains 'mydomain.net' then the addresses will be mapped to 'root@raspberry.mydomain.net' and 'pi@localhost'.

If the file /etc/nullmailer/defaultdomain is empty, then the domain in /etc/mailname is used.

As an example of how the defaultdomain and defaulthost files work together:

defaulthost         myrasberry
defaultdomain       mydomain.com
mapping: from to
root                root@myraspberry.mydomain.com
root@other          root@other.mydomain.com
root@other.place    root@other.place


defaulthost         myrasberry.com
defaultdomain       mydomain.com
mapping: from to
root                root@myraspberry.com
root@other          root@other.mydomain.com
root@other.place    root@other.place

/etc/nullmailer/idhost

If you are using an email account such as your Google mail account to send mail, then this file is not essential.

The file allows you to set the content of the suffix that nullmailer uses when creating message id's for each mail sent. By default, the content of /etc/nullmailer/defaultdomain is used. If required you would normally put a fully qualified domain name (not necessarily real) in here. The message id does not matter outside your system but it is visible in the headers of the mail when sent.

/etc/nullmailer/helohost

You only need this file if you are relaying through an smtp smarthost that won't accept your default domain as a valid mail server. By default, nullmailer uses the value from /etc/mailname for the HELO message in the smtp protocol. Any domain set in this file will be used in the HELO message instead.

/etc/nullmailer/pausetime

This config file may prove particularly useful for your setup on the Raspberry Pi. It contains the  number  of  seconds to pause between successive queue runs when there are messages in the queue (defaults to 60 if this file not present). You should always create this file and enter some value. If you want to retry failed email I would recommend waiting an hour between retry attempts ( enter 3600 in this file ).

However, if  this is set to 0, nullmailer will exit immediately after attempting to send mail once (one-shot mode). This may be desirable if your Pi is only intermittently connected to the network. Without a network connection nullmailer will by default keep attempting to send your mail once every 60 seconds and will write entries for every attempt in your system logs.

By putting the value 0 in /etc/nullmailer/pausetime, nullmailer will attempt to send each mail added to the queue just once. You must attempt to resend queued mail manually.

Any user can trigger an attempt to send all mail waiting in the queue manually at any time by writing a single byte to the /var/spool/nullmailer/trigger pipe. For example:

echo 1 > /var/spool/nullmailer/trigger

will cause nullmailer to attempt to send all messages in the queue.

/etc/nullmailer/sendtimeout

You don't need to set this file unless you have a very specific requirement. The file contains the  number of seconds to wait to complete sending a message before killing the attempt and trying  again (defaults to 3600 if this file not present).  If this is set to 0, nullmailer will wait forever for messages to complete sending (probably a very bad thing).

Summary

If you are using an email account such as your Google mail account to send email, then much of the configuration concerned with address munging does not really matter to you. You may only need to configure /etc/nullmailer/remotes to set your ISP server and /etc/nullmailer/adminaddr if you want local mail forwarding somewhere. Conversely, if you are communicating with a smart smtp relay then applying address mapping and identification settings that result in routable email addresses is probably essential. Hopefully nullmailer can be configured to suit your needs.

In either case, the capability to switch on manual queue delivery using /etc/nullmailer/pausetime should prove useful for many projects where the Pi has an intermittent network connection.

Saturday 23 March 2013

HiPi Modules 0.26

Version 0.26 of the HiPi Modules has been released.

It contains some minor enhancements. I found that some devices with software reset commands don't complete the conversation when 'reset' is sent and leave the caller reporting an error. I added commands to the appropriate device modules that handle this.

There was also a bug in the baudrate setting method for bcm2835 based I2C. The baudrate was always applied to I2C-1 even when I2C-0 was specified. This is now fixed and would appear to allow running two I2C buses at differing baud rates.

Tuesday 19 March 2013

I2C Repeated Start for All

Note : 04 April 2016 : I2C repeated start transfers have been supported by the kernel device drivers for some time. Explicit support has been added to the HiPi modules with Release 0.46 

My last post on the topic of repeated starts and i2c - I hope.

It turns out that some folks actually use languages other than Perl on their Raspberries.  So I've provided easy access with a command line utility as part of the distribution that can be called from Shell, Python, Scratch etc.

It's a setuid executable ( to have the permissions necessary to map specific parts of /dev/mem ) and installs with execute permissions limited to the i2c group. So any user in the i2c group can now use this method of communicating with an i2c slave if required.

Links:


Edit:
It seems possible to control the speeds of  i2c-0 and i2c-1 independently. It will have to wait for me to solder some connections to a Pi for confirmation.





Monday 18 March 2013

i2c and repeated start

Note : 04 April 2016 : I2C repeated start transfers have been supported by the kernel device drivers for some time. Explicit support has been added to the HiPi modules with Release 0.46 

The module version 0.22 had some timing issues with the hardware ( susceptible to hangs under high load ) .

HiPi Modules version 0.23 has been released with appropriate fixes.

For info I have three main i2c devices that I have used for testing the latest release. The MCP23017 port extender has been tested at 1MHz, 400Khz, 100KHz and 32KHz with the system under both heavy and light load using full 22 byte reads and writes.

The MPL3115A2 pressure and temperature sensor that requires the repeated start operation has been tested at 1MHz, 400Khz, 100KHz and 32KHz with the system under both heavy and light load using 24 byte reads and single byte writes.

I also have an ADC from UK supplier HobbyTronics that employs clock stretching. I need to run the bus at 32KHz for this controller to work but it too has no problems at that speed.

It all seems good now.

Sunday 17 March 2013

Raspberry Pi and Repeated Start I2C

Note : 04 April 2016 : I2C repeated start transfers have been supported by the kernel device drivers for some time. Explicit support has been added to the HiPi modules with Release 0.46 

The MPL3115A2 pressure and temperature sensor from Freescale looked like an interesting thing to connect to my Raspberry, so an order was made and the new toy awaited.

I should've Googled first. The sensor requires a particular sequence in the i2c communication protocol known as a repeated start that the i2c device driver on Raspberry doesn't support. Google, consulted belatedly, seemed to confirm it couldn't be done.

Many hours of manic manual reading and code tinkering later, HiPi Perl Modules 0.23 is released including support for repeated-start reads and a HiPi::Interface::MPL3115A2 module to demonstrate the basics.

The answer lay in the BCM2835 manual and its instruction on how to use 10 bit addressing on the i2c bus. To do this you require a repeated start and the manual provides a sequence to achieve this.

Hurrah!

Now my Pi is complete with accurate pressure readings.


The code is straightforward enough if using the provided wrappers. A quick browse of the source should make it clear how to code your own custom applications.

The code that produces the above output is:


#!/usr/bin/perl

use strict;

use warnings;

use HiPi::Interface::MPL3115A2;
# at this point HiPi::BCM2835 is already
# loaded as a dependency but we'll be
# explicit as this is an example
use HiPi::BCM2835;
# for permissions action
use HiPi::Utils;

# let user know if they need sudo
if( $< ) { die 'script must be run using sudo'; }

my $username = getpwuid($<);
print qq(Start up user $username\n);

# initialise the library early then reduce our permissions.
my $targetuser = getlogin() || 'pi';
HiPi::BCM2835->bcm2835_init();
HiPi::Utils::drop_permissions_name($targetuser, $targetuser);

$username = getpwuid($<);
print qq(Now running as user $username\n);

# do work with data
my $dev = HiPi::Interface::MPL3115A2->new;

my $devid = $dev->who_am_i;
print qq(Device ID   $devid\n);

my ( $alt, $pre, $tem ) = $dev->os_all_data() ;
print qq(Temperature $tem\n);
print qq(Pressure    $pre\n);
print qq(Altitude    $alt\n);


The module for HiPi::Interface::MPL3115A2  uses the module wrappers for Mike McCauley's bcm2835 library.  HiPi::BCM2835 and HiPi::BCM2835::I2C need root permissions for read write access to /dev/mem. We can reduce the security problem somewhat by reducing permissions to a less privileged user as demonstrated in the code immediately after initialising HiPi::BCM2835.

So, frustrated Pi owners with dormant MPL3115A2 devices, rejoice - Perl has saved the day.

The modules now have 3 different ways to access I2C, smbus, i2c and  the bcm2835 method shown here. Overkill but I had wrapped them anyway in an attempt to discover the secret of repeat starts.

Install instructions for modules are here.

If you were wise enough to install version 0.21, then upgrade is simple:

hipi-upgrade

If you want to browse the source, version 0.23 should be the default on CPAN soon.

The necessaries have also made it into version 1.24 of the C bcm2835 library released today ( see bcm2835_i2c_read_register_rs ).

Tuesday 12 March 2013

Modules and GUI App Released

I've recently returned to tinkering with Perl on the Raspberry Pi and have published the results as a set of modules and a GUI application.

Unlike my previous effort I've abandoned the idea of a whole distribution of Perl containing the modules I required. The new module distribution is designed to be installed in the standard system Perl on Raspbian Wheezy and installs nearly all of its dependencies using apt-get.

The only exceptions  are the Perl GUI modules. As the current maintainer of wxPerl  I wanted the latest and working installation of Wx and wxWidgets. This isn't available in current Debian based distributions so I've provided pre-built binaries that are installed using PAR::Dist.

I'm quite new to coding access to device drivers and obviously new to accessing the hardware directly so having figured out how to do what I wanted to I decided to wrap up controlling the GPIO and device settings in a GUI.

I found some nice detail about board revisions on the Raspberry Pi Wiki and combined that with the content of /proc/cpuinfo to populate the first tab of my application.


Now I can wonder why my BogoMIPS seem to have reduced to a fixed value across my Model A and my Model B board. On balance I think running apt-get is a good thing but I wish it hadn't stolen 30% of my BogoMIPS. :-)

Onwards to controlling the GPIO pins directly using the bcm2835 library. I implemented just about everything available in the bcm2835 library for the second application tab and then put a few guards in place for combinations of actions that succeeded in hanging my Pi.


It was a bit indulgent implementing a specific GPIO pad control, but there was PAD 5 to think of and the opportunity of code reuse. It gave me a surprising degree of satisfaction switching the peripheral function assignments on and off and seeing the results populate my 'PAD Control'.

Reading the Broadcom information sheets, I found that it isn't possible to query the current state of the pull up and pull down resistors. The settings persist across reboots so it you play at applying resistors to see your input pins go high /low, remember you'll need to set them back at some point.

I'm also now pondering whether to take a soldering iron to a precious Pi to connect PAD 5.


Activating the peripheral function for I2C 0 assigns a couple of pins on this pad.

I took on the GPIO device driver next which exposes the interesting facility to use kernel interrupts with the GPIO pins.  It also offered the promise of having a setup that does not require running with root permissions to control the pins. This introduced me to udev rules and the documentation here. Searching the Raspberry Pi forum was also productive.

The next tab brought this together.


It writes or removes udev rules that give read / write  permissions to members of the group 'gpio' on the gpio device files. I added a button to allow easy management of the group. It is important for any interface that it supports attaching an LED to a pin and allows toggling  it on and off. This is no exception.

I found that changes made in the GPIO Device tab were reflected in the GPIO Pad tabs. However, changes made in the GPIO Pad tabs were not picked up by the device driver. It seems reasonable that should be the case.

Armed with my new knowledge of udev rules it was time to implement the tab for the I2C device.


And lo it was found that in the current Raspbian Wheezy the I2C driver already implements read write access for the group 'i2c'. All I needed to implement was a checkbox to load / unload the kernel modules and code to read and write some configuration files for the 'baudrate' kernel option.

I have a couple of I2C peripherals that seem to employ a communication method called 'clock stretching'. In short, it is a method by which a slave device (my peripherals) can tell the master ( the Raspberry Pi ) to hang on a bit while the slave prepares to reply to a request. It seems the combination of the i2c device driver and Raspberry Pi hardware ( I'm not sure which bit is responsible ) do not currently support 'clock stretching' and the Raspberry Pi ignores the slave's request to wait. However, there's an easy workaround as the device driver supports an option, 'baudrate', which is used to set the I2C bus clock rate. The default value of 100000 prevents some of my peripherals from working, but with a value of 32000, all is well.

It may not be an ideal solution. Everything on the bus is slowed down. For my usage though, it suits well. The settings made in the tab are written to configuration files so apply to any modprobe load / reload and across reboots.  As the default device driver settings apply permissions for the group 'i2c', the option to switch that on and off is disabled.

So, controlling the SPI device ought to be virtually a copy of the procedure for I2C. And so it proved.


The SPI device driver does not automatically give a group read / write permissions on its devices so I needed udev rules here. Because the SPI driver produces device files in /dev, the udev rules were much simpler here than for the GPIO driver. The SPI driver also exposes an option 'bufsiz' so I provided the opportunity to set a value for that. It's unlikely you'll need to alter the default 4096 value.

The settings made in the tab are written to configuration files so apply to any modprobe load / reload and across reboots.

Finally I have a 1 Wire temperature sensor so the HiPi modules include a thin wrapper for kernel device driver. The tab for 1 Wire just lists the attached slave devices and allows loading or unloading of the kernel modules.


And that's it. The GUI was just a fun exercise as a way of implementing something useful without needing any specific attached peripherals or circuits. There are some interesting modules in the distribution that provide high level wrappers for communication with some common ADCs, DACs and the MCP23017 extender ( it's greeeaaat !). There are also a couple of 'LCD control by serial' implementations.

My hope is that the GUI is interesting enough to download and try. Then I'll have captured ye all for the Gods of Perl  when you start making your own improvements and additions.