|
In order to achieve lock-on the system is required to locate the receiver as its position is unknown. A number of methods were looked at. Previous work focused on using a laser and modulating the position at each point, and this had been proven however in this project it was chosen to investigate a different way.
Inspiration came from the Nintendo Wii games console which uses a static array of IR LEDs and an infrared camera with built in object tracking. It was decided to mount a static IR LED on the receiver and use a camera with image processing to locate the IR LED which would act as a ‘homing beacon’ signalling to the transmitter where it is. The reason an IR LED was chosen is due to the fact it will be easier to detect in a lit room. The IR LED was mounted on the receiver with a variable resistor in series with the LED. This is due to the fact that the 5V supply will blow the LED and if powered off a microcontroller will draw too much current. It was decided to make the resistor variable so that the brightness of the LED could be adjusted.
To locate the LED an infrared camera was required. Professional IR cameras are extremely expensive; therefore it was decided to create a custom one for this project. Normal CCD based colour cameras are extremely sensitive to IR light although most come with a filter on the lens to block out the IR. What is required in this project is the opposite, a lens which passes infrared and blocks visible light. Within the photography industry is it fairly well known that a great IR pass filter is unexposed developed photographic film[xxvi]. Many people have used this technique by mounting the film on the back of a lens for taking IR pictures with computer webcams.
Initially experiments were carried out using an old webcam however immediately it was clear the driver was very buggy and extremely slow to process the stream. This meant that as the pan and tilt head scanned an area, the image became blurred and needed extra time to stabilise. Therefore it was decided to use a colour CCD camera and a capture card to interface with a PC. To make the IR filter for the camera some unexposed photographic film was glued to the back of a lens and screwed into the camera.
Figure 29 - IR & Visible Lenses
By deciding to use a camera, a new problem was created. There would have to be some sort of image processing in order to calculate if and where the led is in the images that are being streamed. It was briefly investigated how to capture images in C++ from a video stream however this was abandoned at it was discovered Matlab comes with two extremely useful function toolboxes, the image acquisition and image processing toolboxes.
Using these toolboxes it is possible to capture images from the source and extract all the data and perform calculations on the image content. Due to processing speeds the fastest Matlab can capture pictures is approximately 0.25 seconds. After research and reading the documentation that comes with the toolboxes an example program with code was found[xxvii] which would search for a red laser pointer on a wall and display its coordinates.
The programs main function was named “findlaser.m”. This function was passed a 3 dimensional matrix containing the captured image. And would return the col and row the laser was found in along with a binary image of the suspected locations of the laser. Should the laser not be found the col and row will be NaN or not a number. The function originally took a colour image and extracted the red plane as the laser was red. Since our images are effectively black and white the red, white and blue planes will all be approximately the same. The algorithm works by:
- Inspecting the red plane and finding the highest value present
- Creates binary images of all locations this value is present
- Performs a blob analysis to find the largest connected component in the binary image. The centroid of this component is considered to be the location of the laser, given in pixel coordinates.
- Checks that the number of pixels is greater than a threshold to eliminate noise.
This function was used without modification in order to locate the IR Beacon within the images captured from the camera with IR lens. The only change was a name change of the file and a number of variables.
Figure 30 - Captured IR Image (Beacon Located)
To capture an image first it is necessary to declare and open the stream to the image acquisition device installed on the computer. A list of these devices can be found by typing “imaqhwininfo” at the Matlab command window. To declare a stream ‘vid’ to the ‘winvideo’ device the following command is used:
vid = videoinput('winvideo');
By default any stream or image is in YCbCr (Luminance, chrominance-blue, chrominance-red) format however for most applications RGB (Red Green Blue) is more appropriate. The following command sets the colours to RGB:
set(vid,'ReturnedColorSpace','rgb')
The stream is now created and setup. The live feed can be previewed by calling the command “preview(vid)” or an image can be captured using the next command which captures an image and stores it in the variable frame:
frame=getsnapshot(vid);
This image can be previewed by calling the command “imshow(frame)”. There is another method used to grab frames from the stream using triggers which is used in this project. This simply starts the video, and calls a trigger every time you want to capture a frame. This frame can then be collected.
To detect if the IR beacon is visible within the frame, the frame is passed to the findbeacon function. This function determines all the locations where it thinks the laser could be, and returns the coordinates of the largest region. The command used is:
[col, row, sightings] = findbeacon(frame);
Col is a (1,1) matrix containing the column of the centre of the largest area, and Row again is another (1,1) matrix containing the row. Sightings is a binary matrix of same dimensions as the captured image, and contains all the places in the image where the brightest value was found. An example is shown below.
Figure 31 - Captured Image (Left), Sightings (Middle) & Position Plotted (Right)
The Controlling program was written in Matlab. Its purpose is to capture images from the camera and communicate with the FPGA by sending controlling commands. The captured images are 640x576 pixels.
First it scans the area in a right to left scanning pattern, in a step size of 20 servo positions. When it reaches the end of a line, the camera is moved down a step and the scanning direction is changed. This continues until either the beacon is located or if it reaches the end of the possible travel, the program will quit with a meaningful error message.
If the beacon is located then the second stage involves moving till the IR beacon is in the centre of the screen. It performs this by detecting which quadrant the Beacon is in and then performing a move in the opposite direction since the (0,0) position of the pan and tilt head is in the top right. So to bring the object to the right of the image a X++ command is performed. The centre is defined as a 20 pixel strip in the centre of the image. A important note is that the coordinates in the image are Matlab coordinates (row,col) not Cartesian (x,y) coordinates.
Figure 32 - Regions Of Captured Image
The final stage is transmitting the data to the receiver. Matlab sends a command to the FPGA to enable the modulator. It then checks to see if the beacon has gone low by examining the light intensity at the coordinate of the known position of the beacon. When it goes LOW it counts the length of time elapsed. If the count corresponds to approximately 2 seconds, then the transfer is assumed to be successful.
Figure 33 - AreaScan Flowchart
//Locate Beacon roughly
Initialise variables
Open Serial Port & Video Stream
Loop {
Move to x and y position
CaptureFrame
FindBeacon
If (Beacon is Found) Break from loop
Else {
If (direction = UP) {
If ( a step would exceed x limit){
Change direction
Move a step in y direction
} else { Move a step in x direction }
} else if (direction = DOWN) {
If ( a step would exceed x min){
Change direction
Move a step in y direction
} else { Move a step in -x direction }
}
}
}
//Get Beacon in centre of screen
Capture frame
FindBeacon
Loop {
If (Beacon => the defined centre area) exit loop;
If (Beacon is on left of centre strip) {
Send Increment x servo position by 3 command
Send perform move command }
If (Beacon is on right of centre strip) {
Send Decrement x servo position by 3 command
Send perform move command }
If (Beacon is above centre strip) {
Send Decrement y servo position by 3 command
Send perform move command }
If (Beacon is below centre strip) {
Send Increment y servo position by 3 command
Send perform move command }
Capture Frame
FindBeacon
}
// Communicate
If (Beacon not found) quit function
Calculate Modulation data command
Send Modulation data command
Send EnableModulator command
Loop{
Capture frame
Inspect pixel at last known position
If (pixel = 0) break from loop
}
Send DisableModulator command
Count=0
Loop{
Capture frame
Inspect pixel at last known position
If (pixel = 1) break from loop
Increment Count
}
If (loop executes less than 100 times) “Data successfully received”
Else “No Acknowledge”
Tidy Variables & Exit
FindBeacon function
|
Extract red frame from the passed frame
Find Maximum value of pixel in red frame
Create sightings matrix:
1 : if pixel is greater than 80% of max value
0 : where it is less
Create matrix containing sizes of connected areas
Find index of largest area
If (size of area > threshold) Set Col & Row of this area
Else Col&Row = Not A Number
|
|
Publications
