Raspberry PI tutorial SD slurping

posted on 2013-02-03

This is a tutorial on getting your Raspberry PI to download files from an inserted SD card automatically.

The set up is simple: a Raspberry PI, an USB card reader, and an USB disk to store everything. As I decided to give Arch Linux a try, some parts are Arch Linux specific.

First I checked wether the Card Reader was working with lsusb and cat /proc/partitions:

major minor  #blocks  name

 179        0   30760960 mmcblk0
 179        1      92160 mmcblk0p1
 179        2   30667776 mmcblk0p2
   8       64  976729088 sde
   8       65  976728064 sde1
   8       32     495488 sdc
   8       33     495313 sdc1

The last two partitions where added when I inserted an SD card. So everything works and all drivers are loaded. Mounting /dev/sdc1 will give me access to the files:

mkdir /mnt/sd
mount /dev/sdc1 /mnt/sd

Now to get on to the udev sudo udevadm info -q all --attribute-walk /dev/sdc1 will print out attributes we can use in our udev rule. For some reason my Raspberry PI seemed to crash half-way during this output and I had to reset it. But I got some of the output and decided to create the following rule in /etc/udev/rules.d/33-sd-slurp.rules:

ATTRS{model}=="*SD", ATTR{partition}=="?", ACTION=="add", RUN+="/opt/bin/sd_slurp.sh"

and for testing I created /opt/bin/sd_slurp.sh:

#!/bin/bash
/usr/bin/date >> /tmp/test.txt

To test the udev rule, I reloaded the whole deamon (at one point I nolonger trusterd udevadm control --reload) using the following commands:

sudo systemctl restart systemd-udevd.service
sudo systemctl status systemd-udevd.service

After you have confirmed the rule is working by checking the output in /tmp/test.txt you can go on to creating a python script at /opt/bin/sd_slurp.py:

#!/usr/bin/python2
import os

if __name__ == "__main__":
    f = file("/tmp/hello.txt", "a")
    f.write(str(os.environ))
    f.close()

Now placing the Python script in place of the SH script in udev worked. But as soon as I decided to start copying file in the script it would be cut off half-way. I then found out from the udev manual:

Add a program to the list of programs to be executed for a specific device. This can only be used for very short running tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Long running tasks need to be immediately detached from the event process itself. If the option RUN{fail_event_on_error} is specified, and the executed program returns non-zero, the event will be marked as failed for a possible later handling.

Which is why the program gets cut off. I set of on a journey of daemonizing, double forking and nohup-ping. But none of these seemed to work. Then I found a page hinting that at would be a good solution. So I installed at:

sudo pacman -S at
systemctl enable atd.service
systemctl start atd.service

So the new content of /opt/bin/sd_slurp.sh:

#!/bin/bash
/usr/bin/echo /opt/bin/sd_slurp.py | /usr/bin/at now

This means that our Python script can now take all the time it needs. For logging I use syslog, so all the output of the script should end up in /var/log/user.log and/or /var/log/messages.log. You can find the full source here.

Do not forget to make the both scripts executable using chmod a+x /opt/bin/*

Configuring sd_slurp.py

Two global variables decide where the files are mounted on and copied from whe you insert the card:

MOUNT_POINT="/mnt/sd"
STORAGE_ROOT="/mnt/usb/slurp"

On /mnt/usb I have an USB disk mounted with enough space to store the SD card contents. Afther copying the files, the SD card is unmounted again.

To make sure you don't end up with a large collection of duplicate files, you can run fdupes on the storage point, but that only works if you have your USB disk formatted with a sophiticated enough filesystem.

That's it, you should have it all up and running. Best of luck!