Vision based Line Follower
By Jesper H. Christensen
Contents |
Hardware needed
For installing and utilizing the line follower application for RegBot robots the following things are needed:
- A fully functional RegBot programmed with latest version of the robot software. Consult this section to ensure that.
- A clean Raspberry Pi (model 2 and above is preferred). In case the Pi is clean with an OS newly installed simply skip the install OS section in Software Installation.
- A Raspberry Pi camera modul.
- A mount for Raspberry Pi + camera module. Consult JCA for this.
- A few female/female jumper wires for connecting the Raspberry Pi with the RegBot.
- A WiFi dongle e.g. EDIMAX EW-7811Un (Raspberry Pi model 3 includes WiFi).
Also it would be good to have access to a "HDMI input"-screen in order to locally make the initial connection to WiFi and setup SSH for remote access.
Software installation
Instructions on how to install the needed software on a Raspberry Pi.
Raspberry Pi
Installing the OS
Use this link to download and install the newest Raspian OS on a SD card. Detailed instructions on how to write an image to a SD card is found within the link as well.
When the OS has been successfully installed insert the SD into the Raspberry Pi - connect a screen via the Pi's HDMI output and a keyboard + mouse set.
Setting up WiFi
Plug in the WiFi dongle and open up a terminal window.
In terminal open the wpa-supplicant
configuration file:
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
Go to the bottom of the file and enter the information about the WiFi:
network={ ssid="Your_SSID" psk="Your_wifi_password" }
Save the file by pressing Ctrl+X then Y, then finally press Enter.
Restart the Pi by typing sudo reboot
and it should automatically connect to WiFi when finished restarting.
Setting up SSH
Make sure SSH is enabled. In terminal type:
sudo raspi-config
Scroll down to the "ssh" option and enable it.
Change the password while at it to e.g. filippa corresponding to the RegBots name - default password is raspberry.
Get the Pi's IP address by typing sudo ifconfig
.
Next to the wlan0 entry the inet address is present which is the IP address of the Raspberry Pi, and remote access to the device is available by typing:
On Mac and Linux:
terminal: ssh pi@ip_addr
On Windows:
putty: choose SSH --> pi@ip_addr
By now no screen, mouse or keyboard is needed connected for utilizing the Raspberry Pi.
Assign .local domain to Raspberry Pi
The Raspberry Pi's IP address will frequently change and this will result in spending a lot of time looking up IP addresses of devices. Instead one possibility is to assign a static IP to the device or a .local address to the device, which is the option explained here.
For Linux and Apple computers this should work right out of the box while Windows-users have to download Apple's Bonjour client (note that if Windows-users have iTunes installed then Bonjour is most likely installed as well, as it comes with iTunes). The Bonjour client can be downloaded here.
While still in terminal on Raspberry Pi install Avahi:
sudo apt-get install avahi-daemon
After the installation is done remote access is available by typing:
ssh pi@hostname.local
Default hostname is raspberry. Change the hostname to RegBotXX corresponding to the RegBots number by following these instructions.
Mount remote folder
Working on applications that needs to compile and run locally on the Raspberry Pi it is very handy to mount the working folder locally on a PC or laptop. Doing this will make it possible to develop the code in a preferred IDE, and drag and drop files to/from the device.
On Mac and Linux go to terminal and type:
sshfs pi@hostname.local:/remote/folder ~/local/folder
Remember to create the local folder before mounting to it.
Network Bootstrapper
In order to remotely SSH into the device, the Pi and computer needs to be on the same network. In case the Pi does not know any of the available WiFi networks it will stay unconnected and SSH is not possible.
To overcome this a network bootstrapper must be implemented. Basically all it does is create a AD-HOC network if the Pi does not know or cannot connect to any of the available networks. Connecting the computer to this AD-HOC network opens up SSH access and will allow communication. Use this to put in information of any of the available WiFi networks, or continue as normal on the AD-HOC.
To do this start by installing and configure DHCP Server:
sudo apt-get update sudo apt-get install isc-dhcp-server
The service will start automatically right after installation and will give an error like:
"Not configured to listen on any interface"
Configure some defaults for the DHCP server:
sudo nano /etc/default/isc-dhcp-server
Only change needed:
# On what interfaces should the DHCP server (dhcpd) serve DHCP requests? # Separate multiple interfaces with spaces, e.g. "eth0 eth1". INTERFACES="wlan0"
Save the file by pressing Ctrl+X then Y, then finally press Enter.
Now edit the config file for the DHCP server:
sudo nano /etc/dhcp/dhcpd.conf
Edit it to look similar to this:
DHCPDARGS=wlan0; #args for the dhcpd daemon -> limit DHCP to the wlan0 interface default-lease-time 600; max-lease-time 7200; option subnet-mask 255.255.255.0; option broadcast-address 10.0.0.255; option domain-name "RegBotxxNet"; option routers 10.0.0.1; #default gateway subnet 10.0.0.0 netmask 255.255.255.0 { range 10.0.0.2 10.0.0.20; #IP range to offer } #static IP-assignment host myLaptop { hardware ethernet 11:aa:22:bb:33:cc; fixed-address 10.0.0.100; }
Save the file by pressing Ctrl+X then Y, then finally press Enter.
Now update the interfaces config file:
sudo nano /etc/network/interfaces
Should look similar to this:
# start interfaces upon start of the system auto lo wlan0 # register loopback interface iface lo inet loopback # use dhcp and allow interface to be started when kernel detects a hotplug event allow-hotplug eth0 iface eth0 inet dhcp # use manual ip configuration for wlan0 interface and allow hotplug as well allow-hotplug wlan0 iface wlan0 inet manual
Save the file by pressing Ctrl+X then Y, then finally press Enter.
Next up is to edit the rc.local-file. This script is running every time the Raspberry Pi boots:
sudo nano /etc/rc.local
This scripts will be using bash as the interpreter, so the first line of the file needs to be changed to:
#!/bin/bash
Now go to the bottom of the script, but before the exit 0 line, and add:
# RPi Network Conf Bootstrapper createAdHocNetwork(){ echo "Creating ad-hoc network" ifconfig wlan0 down iwconfig wlan0 mode ad-hoc iwconfig wlan0 essid RegBotxxNet #SSID ifconfig wlan0 10.0.0.200 netmask 255.255.255.0 up /usr/sbin/dhcpd wlan0 echo "Ad-hoc network created" } ssids=( 'MyWlan' 'MyOtherWlan' ) connected=false for ssid in "${ssids[@]}" do if iwlist wlan0 scan | grep $ssid > /dev/null then echo "First WiFi in range has SSID:" $ssid echo "Starting supplicant for WPA/WPA2" wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf > /dev/null 2>&1 echo "Obtaining IP from DHCP" if dhclient -1 wlan0 then echo "Connected to WiFi" connected=true break else echo "DHCP server did not respond with an IP lease (DHCPOFFER)" wpa_cli terminate break fi else echo "Not in range, WiFi with SSID:" $ssid fi done if ! $connected; then createAdHocNetwork fi
Save the file by pressing Ctrl+X then Y, then finally press Enter.
As the DHCP only needs to be started when created a AD-HOC network type:
sudo update-rc.d -f isc-dhcp-server remove
Finally reboot the system and assuming everything succeeded without any errors the Raspberry Pi will connect to any known network, and if no known networks is available the device will automatically create a AD-HOC network.
sudo reboot
OpenCV 3
This guide will give instructions on how to get OpenCV 3 installed and working on the Raspberry Pi.
If not already SSH into the device.
Start by update and upgrade the Pi:
sudo apt-get update sudo apt-get upgrade sudo rpi-update
sudo reboot
Install dependencies
When rebooting is complete SSH into the device again and install some developer tools:
sudo apt-get install build-essential git cmake pkg-config
Image I/O packages:
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
Video I/O packages:
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev sudo apt-get install libxvidcore-dev libx264-dev
GTK development library to compile the highgui sub-module of OpenCV:
sudo apt-get install libgtk2.0-dev
Dependencies:
sudo apt-get install libatlas-base-dev gfortran
Python header files (in case further development or prototyping should take place in python):
sudo apt-get install python2.7-dev python3-dev
OpenCV source code
Get OpenCV 3.0.0 from github and extract in root of Pi:
cd ~ wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.0.0.zip unzip opencv.zip
Get the opencv_contrib repo as well:
wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.0.0.zip unzip opencv_contrib.zip
Compile and install OpenCV
cd ~/opencv-3.0.0/ mkdir build cd build cmake -D CMAKE_BUILD_TYPE=RELEASE \ -D CMAKE_INSTALL_PREFIX=/usr/local \ -D INSTALL_C_EXAMPLES=ON \ -D INSTALL_PYTHON_EXAMPLES=ON \ -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.0.0/modules \ -D BUILD_EXAMPLES=ON ..
Now that the build is all setup, next thing is to compile OpenCV:
make -j4
-j4 stands for the number of cores to use and even when using full 4 cores this can take a very long time.
Assuming no errors occurred when compiling, install it to the system:
sudo make install sudo ldconfig
Now OpenCV 3.0.0 should be installed on the Raspberry Pi system and ready to use!
Line Follower application
Here are instructions on how to get the Line Follower application up and running on the Raspberry Pi.
Raspberry Pi Camera Module API
First off the RaspiCam API needs to be downloaded.
Get raspicam-0.1.3.zip from sourceforge:
cd ~ curl -L -k http://sourceforge.net/projects/raspicam/files/raspicam-0.1.3.zip/download > raspicam-0.1.3.zip unzip raspicam-0.1.3.zip cd raspicam-0.1.3 mkdir build cd build cmake ..
Finally compile, install and update the ldconfig
make sudo make install sudo ldconfig
Line Follower source code
Grab the latest version of the Line Follower source code and compile:
cd ~ git clone git://github.com/JesperChristensen89/LineFollower.git cd LineFollower/build/ cmake .. make
Hopefully no errors should have occured, and the application was created successfully.
Autorun line follower application
In order for the application to run automatically every time the system reboots a mix of tmux and Crontab is used. This will make it possible to SSH into the running instance of the application.
Get tmux:
sudo apt-get tmux
Open up Crontab to make the application run at every boot:
crontab -e
Insert the line:
@reboot tmux new-session -d -s line; tmux new-window -d -n 'linefollow' -t line:1 'sudo /home/pi/LineFollower/build/line_detection'
To save the file press Ctrl+X then Y then finally press Enter.
Reboot the system:
sudo reboot
RegBot integration
In order for Raspberry Pi to run and communicate with the RegBot it needs to be connected via the jumper-wires.
The Raspberry Pi pins is located as the figure shows:
An overview of the RegBot's PCB is:
Use Pin 2 and Pin 6 to supply power:
- Connect Pin 2 to RegBot Sharp IR pin 2.
- Connect Pin 6 to RegBot Sharp IR pin 1.
Use Pin 8, Pin 10 and Pin 14 for serial communication back and forth to the RegBot:
- Connect Pin 8 to RegBot UART pin 4.
- Connect Pin 10 to RegBot UART pin 5.
- Connect Pin 14 to RegBot UART pin 1.
Turn on the system, and the Raspberry Pi should now be connected through the RegBot and all communications to the RegBot is now wireless through the Raspberry Pi's Line Follower application via SSH.
However when uploading to RegBot a wired USB connection to the Teensy board is still needed.