MIT Self Driving Race Car Ultra Sonic Sensor Test


Hey Steemians , This is another video in the MIT Race Car Series project.
In this Video we carried out a few preliminary tests to test the movements by using depth sensors and Zed Camera.
If you would like to configure your car just like the one you see in the video i would recommend following this Guide .

Hardware Tested In this Video

  • Zed Camera
  • The reason why we use the ZED camera is because it enables Real-time 3D Mapping which is essential in identifying objects and thereby enabling the robot in making smart decisions.You may find some other Camera's which may work just as well but this was the equipment provided to us for completing this project.

    All the Code written for Testing Depth Sensing is included on the Github Repository here .We used Python for implementing these sensors as there are a collection of default Libraries in python which will make our life easier.

    For starter's you will require the following imports to use the PyZedCamera
    import pyzed.camera as zcam
    import pyzed.defines as sl
    import pyzed.types as tp
    import pyzed.core as core

    After importing all the required libraries you can now create an Object of type PyZEDCamera and use all the additional features of PyZedCamera
    zed = zcam.PyZEDCamera()

    Now You may initialize the depth mode and and specify the depth measurements required.
    init_params = zcam.PyInitParameters()
    init_params.depth_mode = sl.PyDEPTH_MODE.PyDEPTH_MODE_PERFORMANCE # Use PERFORMANCE depth mode
    init_params.coordinate_units = sl.PyUNIT.PyUNIT_MILLIMETER # Use milliliter units (for depth measurements)

    Finally here is the code which will allow your robot to assess depth in various images and overwrite the Electronic speed controller thereby enabling your bot to make smart decisions

        # Open the camera
        err = zed.open(init_params)
        if err != tp.PyERROR_CODE.PySUCCESS:
            exit(1)
    
        # Create and set PyRuntimeParameters after opening the camera
        runtime_parameters = zcam.PyRuntimeParameters()
        runtime_parameters.sensing_mode = sl.PySENSING_MODE.PySENSING_MODE_STANDARD  # Use STANDARD sensing mode
    
        i = 0 #counter
        image = core.PyMat()
        depth_for_display = core.PyMat()
    
        print('Current mode: Capture {} images as fast as possible.\nMerge the images.\nSave to pickle files.'.format(num_images))
        while i < num_images:
            # JOYSTICK
            pygame.event.pump() # keep everything current
            throttle = (j.get_axis(throttle_axis)+1)/2 # left stick
            steering = (j.get_axis(steering_axis)+1)/2 # right stick steering
            left_bumper = j.get_button(left_bumper) # left bumper is deadman/ controls whether you are saving
            exit_button = j.get_button(exit_axis) # Options button exits
    
            if exit_button:
                print('Exit button (options) pressed. Stopping data collection')
                break
    
            if left_bumper:
                # A new image is available if grab() returns PySUCCESS
                if zed.grab(runtime_parameters) == tp.PyERROR_CODE.PySUCCESS:
                    # Retrieve left image
                    zed.retrieve_image(image, sl.PyVIEW.PyVIEW_LEFT)
                    # Retrieve left depth
                    zed.retrieve_image(depth_for_display,sl.PyVIEW.PyVIEW_DEPTH)
    
                    #convert to arrays
                    data=image.get_data()
                    depth_data=depth_for_display.get_data()
    
                    # Convert images to smaller square images
                    square_image_size=500
                    data=cv2.resize(data,(square_image_size,square_image_size))
                    depth_data=cv2.resize(depth_data,(square_image_size,square_image_size))
    
                    merged = merge_images(data,depth_data)
                    print('writing dataset/image_{0}_{1:.4f}_{2:.4f}.pickle'.format(i,throttle,steering))
                    if (i%25==0):
                        print('{} images have been captured'.format(i))
    
                    pickle.dump(merged,open( 'dataset/image_{0}_{1:.4f}_{2:.4f}.pickle'.format(i,throttle,steering), 'wb' ))
                else:
                    print('image collection failed')
                # Increment the loop
                i = i + 1
    
        j.quit()
        print('Image capture complete')
        # Close the camera
        zed.close()
    

    Hope you enjoyed this post, feel free to reach out if you would like to replicate this project or have any questions regarding this project.


    ▶️ DTube
    ▶️ IPFS
    H2
    H3
    H4
    Upload from PC
    Video gallery
    3 columns
    2 columns
    1 column
    14 Comments