Devan-BSE-24

View on GitHub

Ball Tracking Robot

This rover uses a Picam to detect a red ball in front of it, moves towards it, and uses ultrasonic sensors to stop it when it gets too close. While I had some trouble with code, I overcame my challenges and completed this robot with my modifications.

Engineer School Area of Interest Grade
Devan G Marin Academy Electrical Engineering Incoming Sophmore

Headstone Image

Final Milestone

My final milestone was the addition of my modification. Because I noticed my robot would not be able to track the ball unless it was in its camera frame, I decided that I wanted the robot to rotate around in a circle. This would make it so that eventually the robot would find the ball and drive towards it. I then added some logic to track the ball, creating a new variable called ball. Ball is set at 0 and if the ball is on the left, ball = 1, if the ball is on the right, ball = 2, and if ball is in the middle, ball = 0. With this information, if the ball goes offscreen because the car ran into it or my dog picked it up, it would rotate in the correct direction instead of flipping a coin and hoping it went the right way. That concludes my modification. Some troubles I had with this was not allowing enough time for the car actually to turn, as the delays were too short. I fixed this by lengthening the delays and adding stops in the right places. While this was not my biggest challenge, like the car overturning, it still negatively affected my results.

Second Milestone

My second milestone was the completion of my base project. I now have a robot that tracks and moves towards a red ball. The first part of this milestone was configuring the Picam to detect the ball. To do this, we had the Picam mask out of all colors besides red. Because the ball is the only red item in the sight of the Picam, it makes the red ball the only thing shown once masked. Once we isolate the ball, we then use a function that finds the largest blob on the screen, which in this case will be the ball, and tells us the location of it. With some < or > signs, we can use where it is placed on the screen to tell how far the car needs to turn so it can drive directly toward it. I had a lot of errors with how far the car would turn, as it would overturn because of the camera’s frame rate being too slow, causing it to detect the ball to much. This was solved by finding the part of the code that took up the most time and fixing it so it wouldn’t cause as many delays. My next milestone will be modifications, which will make my robot turn around looking for the ball when the ball is not detected.

First Milestone

My first milestone is the compilation of all my components. This includes setting up a Raspberry Pi, which I had trouble with. While trying to connect Visual Studio Code to the Raspberry Pi, an error saw me choosing the wrong OS for the Raspberry Pi in the selection menu in Visual Studio Code. While this error was a little annoying, it was worth going through so now in the future I understand what OS my Pi runs. This Pi will run the code that will control the motors in the car, however, it needs an H-Bridge to intermediate the communication between the two. The H-Bridge is essentially a middleman that reads the code from the Raspberry Pi and tells the motors what to do. Connected to the Raspberry Pi is also a camera, called Pi cam, which in the future, will be used to find out where the ball is and communicate back to the Pi that we need to move towards it. While the car is moving towards the ball, it might run into obstacles. For that, we have ultrasonic sensors attached to the car, making sure that if it gets too close to a wall, desk, pet, etc, it will not run into it. The next step will be to add more code so that the car will be able to track the ball and move towards it.

Schematics

Image

Code

import cv2
from picamera2 import Picamera2, Preview
import RPi.GPIO as gpio
import time              
import numpy as np

ball = 0

gpio.setmode(gpio.BCM)
gpio.setwarnings(False)
gpio.setup(17, gpio.OUT)
gpio.setup(22, gpio.OUT)
gpio.setup(23, gpio.OUT)
gpio.setup(24, gpio.OUT)

from gpiozero import DistanceSensor

def distanceSensor():
  ultrasonic1 = DistanceSensor(echo=14, trigger=15, threshold_distance=0.5)
  ultrasonic2 = DistanceSensor(echo=20, trigger=21, threshold_distance=0.5)
  ultrasonic3 = DistanceSensor(echo=5, trigger=6, threshold_distance=0.5)
  return ultrasonic3.distance * 100, ultrasonic2.distance * 100, ultrasonic1.distance * 100
  

def reverse():

 gpio.output(17, True)
 gpio.output(22, False)
 gpio.output(23, True) 
 gpio.output(24, False)


def forward():

 gpio.output(17, False)
 gpio.output(22, True)
 gpio.output(23, False) 
 gpio.output(24, True)


def stop(sec):
 
 gpio.output(17, False)
 gpio.output(22, False)
 gpio.output(23, False) 
 gpio.output(24, False)
 time.sleep(sec)

def sharpRight():
 
 gpio.output(17, True)
 gpio.output(22, False)
 gpio.output(23, False) 
 gpio.output(24, True)
 
def right():
 
 gpio.output(17, False)
 gpio.output(22, False)
 gpio.output(23, False) 
 gpio.output(24, True)
 
 

def sharpLeft():
 
 gpio.output(17, False)
 gpio.output(22, True)
 gpio.output(23, True) 
 gpio.output(24, False)

def left():
 
 gpio.output(17, gpio.LOW)
 gpio.output(22, gpio.HIGH)
 gpio.output(23, gpio.LOW) 
 gpio.output(24, gpio.LOW)

def segment_colour(frame):   
    hsv_roi =  cv2.cvtColor(frame, cv2.COLOR_RGB2HSV)
    
    mask_1 = cv2.inRange(hsv_roi, np.array([100, 190,1]), np.array([190,255,255]))
    
    mask = mask_1 
    kern_dilate = np.ones((12,12),np.uint8)
    kern_erode  = np.ones((6,6),np.uint8)
    mask = cv2.erode(mask, kern_erode)
    mask = cv2.dilate(mask, kern_dilate)
    
    (h,w) = mask.shape
    
    
    
    #cv2.imshow('mask', mask) 
    
    return mask


def find_blob(blob):  
    largest_contour=0
    cont_index=0
    contours, hierarchy = cv2.findContours(blob, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
    for idx, contour in enumerate(contours):
        area=cv2.contourArea(contour)
        if (area >largest_contour):
            largest_contour=area
            cont_index=idx
                    
    r=(0,0,2,2)
    if len(contours) > 0:
        r = cv2.boundingRect(contours[cont_index])
        
     
    return r,largest_contour


picam2 = Picamera2()

camera_config = picam2.create_still_configuration(main={"size": (1920, 1080)}, lores={"size": (480, 270)}, display="lores")

picam2.configure(camera_config)
#picam2.start_preview(Preview.QTGL)  
picam2.start()

time.sleep(2)



while(True):

    distance1, distance2, distance3 = distanceSensor()
    #print(distance1, distance2, distance3)


    startTime = time.time()
    
    if distance1 < 5:
       stop(5)
    if distance2 < 5:
       stop(5)
    if distance3 < 5:
       stop(5)
    
    im = picam2.capture_array()
    height = im.shape[0]
    width = im.shape[1]
     
    mask_red=segment_colour(im[:,:,[0,1,2]])
    loct,area=find_blob(mask_red)
    x,y,w,h=loct
    print(area, loct)
    centerx = x + w/2
    centery = y + h/2
    center = (centerx, centery)
    print(center)
    #if area > 1100000:
        #stop(1)
    if centerx < 200 and centerx > 0 and area > 10000:
        left()
        time.sleep(0.05)
        stop(0.01)
        print("left")
        ball = 1
    elif centerx > 1700:
        time.sleep(0.05)
        right()
        stop(0.01)
        print("right")
    elif centerx > 1600 and area > 10000:
        ball = 2
    else:
        stop(0.01)
        forward()
        time.sleep(0.05)
        print("forward")
        ball = 0
    if area < 100000 and ball == 1:
        stop(0.01)
        sharpLeft()
        print("sharp left")
        time.sleep(0.05)
        stop(0.01)
    elif area < 100000 and ball == 2:
        stop(0.01)
        sharpRight()
        print("sharp right")
        time.sleep(0.1)
        stop(0.01)
    elif area < 100000 and ball == 0:
        stop(0.01)
        sharpRight()
        print("sharp right")
        time.sleep(0.1)
        stop(0.01)
    stopTime = time.time()
    elapsedTime = stopTime - startTime
    #print(elapsedTime)
    print(ball)

Bill of Materials

Part Note Price Link
Raspberry Pi Kit Brains of the operation; sends code to H-Bridge and receives information from the camera and sensors $95.19 Link
Robot Chassis Holds all the components $18.99 Link
Screwdriver Kit Used to screw screws in $5.94 Link
Ultrasonic Sensor Detect the distance from objects $9.99 Link
H Bridges Receives commands from the Raspberry Pi and sends them to motors $7.79 Link
Pi Cam Used to find where the ball is $12.86 Link
Electronics Kit Wires, resistors, and a breadboard for all the components $11.98 Link
Motors To make the car go forward $11.98 Link
DMM To test the voltage of the batteries $11 Link
AA Bateries Power the cards $14.89 Link
Champion Sports Ball The ball that is detected $12.34 Link
USB Power Bank and Cable Power the Pi $16.19 Link