Lab

Recommended Tools

In addition to the resources that the course provides, students may find it useful to obtain the tools below:

  • wire cutter
  • wire stripper
  • needle nose pliers
  • screwdrivers
  • hex set (ball ends)
  • electrical tape
  • small scissors
  • a small file

Lab Resources

The Srivastava Senior Design Lab (2070 ECEB) is dedicated to ECE 445 usage. This lab provides you access to a vast array of lab equipment, hardware, and software for your use in developing and implementing your senior design project. In addition, course staff will make themselves available in this lab during their office hours to provide guidance on your project throughout the semester. It is our intention that this laboratory space provides you and your team with all the tools you would need to develop and test your project (within reason!). If there is something that you require in the lab to complete your project that does not exist in the lab, talk to your TA and we will see if we can remedy the situation.

Lab Bench Reservations

If and when the semester gets so busy that finding a lab bench to work at becomes difficult, please make use of the Lab Bench Reservation System in PACE. Reserving a lab bench will guarantee priority access to that bench, even when the lab is busy. To use the tool, after authenticating in PACE, you will see a page with a title "Signup for lab bench" with some text and a large table below that. The table shows the schedule for each bench on a given day (use the orange arrows or "Go To Date" text box to see a different day).  You make your reservation by simply clicking in a grid cell in the table, which will turn the box green. Click on it again to un-reserve the bench (and the box will turn white again).  Benches that are already reserved by another group will be denoted with a yellow box (you can hover your mouse over a yellow box to find out what group has reserved the bench).

A few ground rules:

  1. You may use a lab bench (a) during a time for which you have it reserved or (b) any time during which it is not reserved in the system (on a first-come-first-served basis). However, if you are working at a bench that is unreserved and somebody reserves it using the online system, the group with the reservation gets the lab bench.
  2. There is a limit on the amount of time for which you can reserve benches in 2070 ECEB.  The limit is currently a total of 4 hours of total bench time in the lab per group per day (e.g., 2 hours at Bench A and 2 hours at Bench B would max out your team's reservations for the day).  While this may seem restrictive, keep in mind that the course serves more than 60 groups in a typical semester and the lab has only 16 benches.  Also keep in mind that you can work at a bench if it is unreserved.
  3. Some lab benches have specialized equipment at them, such as digital logic analyzers.  Try to reserve the lab bench that has the equipment that you need.
  4. Cancel reservations that you will not need as soon as possible to give other groups a chance to reserve the lab bench.  You can cancel a reservation up to 1 hour before time and not have it count against your daily allotment.
  5. Conflicts and/or reports of people not following these rules should be sent to your TA with the course faculty in copy.
  6. Above all, be courteous.  Especially near the end of the semester, the lab will be full most of the time and stress will abound.  Clean up the lab bench when you are done with it.  Start and end your sessions on time.  Be patient and friendly to your peers and try to resolve conflicts professionally.  If we notice empty lab benches that have been reserved, we will cancel your reservations and limit your ability to reserve lab benches in the future. Similarly, do not reserve more time than you will need.  If we notice that you are frequently canceling reservations, we will limit your ability to reserve lab benches in the future. Finally, do not try to “game” the system and reserve a bench for 30 minutes every hour for eight hours. We will notice this and revoke your ability to reserve a bench.

Lab Rules

There are two overriding rules of working in the Srivastava Senior Design Lab. First, be safe. Second, be courteous. Lab access will be revoked if you fail to complete the required laboratory safety training by the deadline or if you break any of the lab rules. Specific points and examples of what we expect:

Breaking the rules or exhibiting bad laboratory etiquette will lead to a loss of points and/or revocation of laboratory access.

Lab Equipment Rules

Do not remove any equipment from the lab. Students may not change the connections on equipment without TA approval. Any approved changes that are made should be undone before leaving the lab. If a bench instrument is malfunctioning, a red repair tag should be placed on it and you should notify your TA. This alerts the staff to the problem, and allows the Electronics Services Shop to fix the problem.

When using a piece of laboratory equipment for the first time, please ask a TA for help. If you are inexperienced with a piece of hardware, do not assume that it is broken just because you cannot figure out how to use it. Similarly, if you use a piece of equipment to test your project and the equipment does not perform the way you think it should, do not assume the fault is with the equipment, and do not try again with equipment on another bench. Rather, stop and make absolutely sure the problem is not with your connections or project.

If you break any laboratory equipment, you must tell your TA within 1 business day. Any attempts to conceal breakage will result in an F in the course.

Room Access

The lab room (2070 ECEB) is on the electronic key-card system. The Department automatically adds room access to the building and the lab for all students on the roster. You will need a “prox enanabled” I-Card to swipe into the room. If the door does not open after several attempts, you may need to get a replacement card. Room access is automatically restricted to faculty and TAs during official breaks (i.e., Thanksgiving, Christmas, and Spring Break).

Computer Access

The lab computers are EWS computers and are setup like other Windows-based EWS systems you are familiar with. Standard EWS rules apply to these machines. In particular, please store any/all files you generate on a network drive or in the cloud. The C: drive should not be used for any personal material, since it is unprotected and is available only on the particular machine where it was originally stored. A particular computer may be cleared and reconfigured at any time for maintenance reasons.

In addition to the desktop computers, EWS maintains the printer in the lab. You are free to use it to print documents related to your project, but be aware that this printing counts against your standard print quota.

Smart Glasses for the Blind

Siraj Khogeer, Abdul Maaieh, Ahmed Nahas

Smart Glasses for the Blind

Featured Project

# Team Members

- Ahmed Nahas (anahas2)

- Siraj Khogeer (khogeer2)

- Abdulrahman Maaieh (amaaieh2)

# Problem:

The underlying motive behind this project is the heart-wrenching fact that, with all the developments in science and technology, the visually impaired have been left with nothing but a simple white cane; a stick among today’s scientific novelties. Our overarching goal is to create a wearable assistive device for the visually impaired by giving them an alternative way of “seeing” through sound. The idea revolves around glasses/headset that allow the user to walk independently by detecting obstacles and notifying the user, creating a sense of vision through spatial awareness.

# Solution:

Our objective is to create smart glasses/headset that allow the visually impaired to ‘see’ through sound. The general idea is to map the user’s surroundings through depth maps and a normal camera, then map both to audio that allows the user to perceive their surroundings.

We’ll use two low-power I2C ToF imagers to build a depth map of the user’s surroundings, as well as an SPI camera for ML features such as object recognition. These cameras/imagers will be connected to our ESP32-S3 WROOM, which downsamples some of the input and offloads them to our phone app/webpage for heavier processing (for object recognition, as well as for the depth-map to sound algorithm, which will be quite complex and builds on research papers we’ve found).

---

# Subsystems:

## Subsystem 1: Microcontroller Unit

We will use an ESP as an MCU, mainly for its WIFI capabilities as well as its sufficient processing power, suitable for us to connect

- ESP32-S3 WROOM : https://www.digikey.com/en/products/detail/espressif-systems/ESP32-S3-WROOM-1-N8/15200089

## Subsystem 2: Tof Depth Imagers/Cameras Subsystem

This subsystem is the main sensor subsystem for getting the depth map data. This data will be transformed into audio signals to allow a visually impaired person to perceive obstacles around them.

There will be two Tof sensors to provide a wide FOV which will be connected to the ESP-32 MCU through two I2C connections. Each sensor provides a 8x8 pixel array at a 63 degree FOV.

- x2 SparkFun Qwiic Mini ToF Imager - VL53L5CX: https://www.sparkfun.com/products/19013

## Subsystem 3: SPI Camera Subsystem

This subsystem will allow us to capture a colored image of the user’s surroundings. A captured image will allow us to implement egocentric computer vision, processed on the app. We will implement one ML feature as a baseline for this project (one of: scene description, object recognition, etc). This will only be given as feedback to the user once prompted by a button on the PCB: when the user clicks the button on the glasses/headset, they will hear a description of their surroundings (hence, we don’t need real time object recognition, as opposed to a higher frame rate for the depth maps which do need lower latency. So as low as 1fps is what we need). This is exciting as having such an input will allow for other ML features/integrations that can be scaled drastically beyond this course.

- x1 Mega 3MP SPI Camera Module: https://www.arducam.com/product/presale-mega-3mp-color-rolling-shutter-camera-module-with-solid-camera-case-for-any-microcontroller/

## Subsystem 4: Stereo Audio Circuit

This subsystem is in charge of converting the digital audio from the ESP-32 and APP into stereo output to be used with earphones or speakers. This included digital to audio conversion and voltage clamping/regulation. Potentially add an adjustable audio option through a potentiometer.

- DAC Circuit

- 2*Op-Amp for Stereo Output, TLC27L1ACP:https://www.ti.com/product/TLC27L1A/part-details/TLC27L1ACP

- SJ1-3554NG (AUX)

- Connection to speakers/earphones https://www.digikey.com/en/products/detail/cui-devices/SJ1-3554NG/738709

- Bone conduction Transducer (optional, to be tested)

- Will allow for a bone conduction audio output, easily integrated around the ear in place of earphones, to be tested for effectiveness. Replaced with earphones otherwise. https://www.adafruit.com/product/1674

## Subsystem 5: App Subsystem

- React Native App/webpage, connects directly to ESP

- Does the heavy processing for the spatial awareness algorithm as well as object recognition or scene description algorithms (using libraries such as yolo, opencv, tflite)

- Sends audio output back to ESP to be outputted to stereo audio circuit

## Subsystem 6: Battery and Power Management

This subsystem is in charge of Power delivery, voltage regulation, and battery management to the rest of the circuit and devices. Takes in the unregulated battery voltage and steps up or down according to each components needs

- Main Power Supply

- Lithium Ion Battery Pack

- Voltage Regulators

- Linear, Buck, Boost regulators for the MCU, Sensors, and DAC

- Enclosure and Routing

- Plastic enclosure for the battery pack

---

# Criterion for Success

**Obstacle Detection:**

- Be able to identify the difference between an obstacle that is 1 meter away vs an obstacle that is 3 meters away.

- Be able to differentiate between obstacles on the right vs the left side of the user

- Be able to perceive an object moving from left to right or right to left in front of the user

**MCU:**

- Offload data from sensor subsystems onto application through a wifi connection.

- Control and receive data from sensors (ToF imagers and SPI camera) using SPI and I2C

- Receive audio from application and pass onto DAC for stereo out.

**App/Webpage:**

- Successfully connects to ESP through WIFI or BLE

- Processes data (ML and depth map algorithms)

- Process image using ML for object recognition

- Transforms depth map into spatial audio

- Sends audio back to ESP for audio output

**Audio:**

- Have working stereo output on the PCB for use in wired earphones or built in speakers

- Have bluetooth working on the app if a user wants to use wireless audio

- Potentially add hardware volume control

**Power:**

- Be able to operate the device using battery power. Safe voltage levels and regulation are needed.

- 5.5V Max

Project Videos