Project

# Title Team Members TA Documents Sponsor
9 Image acquisition, 3D reconstruction and a visual interactive digital heritage system
Chuanrui Chen
Denglin Cheng
Qianyan Shen
Ziying Li
 design_document2.pdf
final_paper2.pdf
proposal1.pdf
 Shurun Tan
Spring 2024 ECE445 RFA

Image acquisition, 3D reconstruction and a visual interactive digital heritage system

# TEAM MEMBERS:

- Qianyan Shen (qianyan2)

- Ziying Li (ziyingl4)

- Chuanrui Chen (cc86)

- Denglin Cheng (denglin3)

# Problem

Cultural artifacts possess significant historical, cultural, and artistic value. However, due to the passage of time and the impact of natural deterioration, many artifacts face risks of damage, loss, or decay. Additionally, for history enthusiasts and researchers worldwide, detailed information about specific artifacts is not readily accessible.

Traditional photographs often fail to capture the intricate details of artifacts, hampering comprehensive research and preservation efforts. Furthermore, the absence of user-friendly interactive interfaces limits the interaction between enthusiasts and artifacts, impeding immersive experiences in virtual exploration of cultural heritage.

Therefore, our team aims to develop a system that can generate realistic 3D models of cultural artifacts and provide users with a user-friendly interactive interface for immersive exploration.

# Solution Overview

Our system will use advanced scanning and 3D reconstruction techniques to capture the detailed geometry of cultural artifacts. This will be achieved through a series of subsystems including a Stabilized Scanning Subsystem, 3D Reconstruction Subsystem, Database Subsystem, and Interactive Interface Subsystem. Please refer to the following subsystem descriptions for more detailed information.

# Solution Components

## Stabilized Scanning Subsystem
This subsystem aims to capture detailed 3D data of the workpiece with high precision and low noise by coordinating a self-stabilizing three-axis gimbal centered around the STM32 microcontroller.
We intend to use solidworks to build the three axis parts of the gimbal respectively, and print them out with a high-precision 3D printer, and then use the brushless motor to connect these parts, and control them with the STM32 code, so that it can achieve real-time angular correction, so that in the process of scanning can be done to achieve the lens anti-shake, reduce motion blur.

## 3D Reconstruction Subsystem
This subsystem aims to obtain a point cloud through RGBD images and perform 3D reconstruction using the point cloud.
We first use a depth camera to capture RGBD images of an object from different angles and preprocess the raw images by denoising and repairing. Then, we proceed with point cloud acquisition, registration, and reconstruction to obtain a 3D model.
To begin, we calibrate the camera to obtain the lens parameters. We then convert the 2D coordinate system of the depth image to a 3D point cloud and map the pixel colors from the RGB image to the 3D point cloud. Afterward, we process the obtained point cloud by applying denoising and sampling techniques, facilitating subsequent registration and reconstruction steps. By repeating these processes, we obtain point clouds from different angles, and we perform precise registration using the ICP （Iterative Closest Point） method to align them in a unified coordinate system. Finally, the 3D reconstruction is completed using the Poisson reconstruction algorithm or other techniques.

## Database Subsystem
This subsystem aims to store the basic information of the artifacts, including dynasties, historical backgrounds, stories, etc., and at the same time saving the generated complex 3D model data.
With database system, users can upload the information of artifacts from all over the world to the database, and can also retrive and view the artifacts from exotic countries. When a user wants to retrieve an artifact, the database will find the corresponding information from its own stored data according to the search item entered by the user and display it through the Interactive Interface Subsystem for users to view artifacts from around the globe.


## Interactive Interface Subsystem
This subsystem aims to provide a user-friendly interface that facilitates database interaction and basic visualization capabilities, delivering a visually pleasing experience to users and catering to their close-range viewing needs.

We aim to present brief introductions of multiple cultural artifacts on the interface, including physical photos, names, dynasties, and more. Upon selection, users can access the corresponding detailed information and the reconstructed 3D model by linking to the database. Specifically, we render the obtained 3D models and offer features such as rotation and scaling for users to observe the artifact's details. Additionally, the interface can include a filtering function to provide users with a certain degree of personalized service in selecting artifacts.

# Criterion for Success
Successfully captures information about the appearance of artifacts without requiring the user to manually adjust examples or angles to minimize the noise.
Accurate and detailed 3D scanning and reconstruction of artifacts.
A database subsystem for effective data management and data retrieval.
A user-friendly interactive interface provides an immersive experience in cultural heritage exploration.


# Divisions Of Labor And Responsibilities
Denglin Cheng is responsible for the modeling of the Stabilized Scanning Subsystem, 3D printing, and the design of the control circuits in the STM32, as well as the final assembly and debugging of the gimbal to ensure smooth scanning of the depth camera.

Qianyan Shen is responsible for RGBD image preprocessing, point cloud acquisition, alignment, and 3D reconstruction.

Ziying Li is responsible for enabling database system to store and retrive data and interact with front-end.

Chuanrui Chen is responsible for the specific design and implementation of the UI interface, requiring her to understand and utilize the database interface. She also assists in the acquisition of point clouds from RGBD images and the design of the control circuits in the STM32.




Miniaturized Breath Sensors

Rui Cai, Yiyang Chen, Qiaozhi Huang, Yingzhuo Wang

Featured Project

## Group Member:

- Yiyang Chen[yiyangc5];

- Rui Cai[ruic2] ;

- Yinzhuo Wang[yw28];

- Qiaozhi Huang[qiaozhi2]

## Problem

Flow monitoring is crucial in many applications. We want to build a miniaturized breath sensor system that can monitor asthma.

## Solution Overview

In this wearable respiratory monitoring device, a new fluid measurement device, similar in principle to a traditional hotline, will be used to collect real-time data on a person's breathing rate. In contrast to the traditional hotline, materials such as graphene and carbon nanotubes are used as probes which is much more robust and have lower TCR(temperature coefficient of resistance). This material--graphene fiber (GF) will be welded into Wheatstone bridge and the voltage output of GF will demonstrate the velocity of air flow by controlling the temperature of the GF. Then, we will use filter to eliminate noise of the signal and do Fourier Transform to demonstrate the frequency of respiration. After that, this signal can be sent to smartphone. With previous training data online, we can analyze the signal of respiration and conclude the probability of asthma. We plan to use a mobile app to show users breathing data, summarize the data and make recommendations. We will use Bluetooth for data transmission.

## Solution Components

### Flow Sensor System

The resistance of a specific material changes at different temperatures, and the flow sensor system's control circuit measures the change in resistance to achieve constant temperature control of the sensor probe. In the thermostatically controlled fluid sensor subsystem, the heat carried by the fluid at different speeds through the sensor probe is the same as the heat provided by the compensation circuit, so that the fluid flow rate can be accurately measured. Graphene and carbon nanotubes are widely used in these sensor probes, and sensor probes using pencil and paper have recently been proposed as a new type of sensor probe. The processing of sensor probes is challenging and there are advantages and disadvantages to various methods, including soldering and metal clamping, and we are trying to design a small, low-cost and robust sensor probe.

### Circuit

The circuit of our design consists of three sections: Wheatstone Bridge, Amplifier, and Feedback control. We need to adjust the resistance of the Wheatstone Bridge to construct and balance a working space for GF sensor. As it states in previous, the flow would change the GF material’s resistance, thus create a voltage difference on both sides of the Wheatstone Bride. This difference will be amplified by the operational amplifier, and the voltage regulator will change the excitation voltage on the Wheatstone Bridge in order to keep the temperature of GF stable. The difficulty of our design come from the feedback control design. One possible way is to use transistors. In addition, if we want to eliminate the environmental temperature effect, specific temperature compensation measure should be implemented, such as add a temperature sensor in another Wheatstone Bridge. The circuit should keep the GF temperature stable and output the voltage change, this output signal will transfer to next section and be processed and analysed.

### Signal Processing and Analysis

First, we must use filter to eliminate noise of signal. As we all know, the high frequency noise can have a negative influence on the signal, which does harm to our analysis of asthma. Therefore, we must do FFT on signal we get from circuit and use high frequency filter to eliminate certain noise. Second, to calculate the probability of asthma, we must collect training data of respiration online. These data can be used to do machine learning. With those training data, the signal can be analysed easily.

### Result

Visualization Bluetooth Low Energy (BLE) features Low power consumption and faster transmission speeds. Therefore, we choose BLE to transmit data to mobile phone on this wearable respiratory monitoring device that requires long battery life and only a small amount of data transfer. We're also going to keep the interface simple and add analysis function to the app.

## Criterion of success

- Wearable and Miniaturized In the current study, wearability and miniaturization directly determine the industrialization potential of this new type of sensor. The portability of the product will help to achieve 24/7 patient health monitoring. Therefore, the development of wearable and miniaturized health monitors is considered as one of the criteria to measure the success of the product.

- Comfortable and Flexible Flexible sensors that conform to human science will significantly improve the comfort of wearing the product and determine the user's willingness to wear it. Flexibility and comfort are one of the goals of the product.

- Environment Friendly Environmental protection is becoming an increasingly important issue to be addressed today. The development of environment-friendly sensors is the goal of this research. Conventional biosensors will inevitably use environmentally hazardous materials such as plastic. this study will use degradable materials, such as paper, instead of plastic for product development.

- Low Cost Low-cost respiratory health monitors facilitate product penetration and daily use.

- Reliable and Stable As a medical product, the reliability of the product determines the safety of the life of the target object. A highly reliable and high-performance respiratory monitoring device can effectively guarantee the occurrence of accidents.

## Distribution of Work

Yiyang Chen (ME), Rui Cai (EE) and Qiaozhi Huang (ME) will be responsible for the construction of the fluid sensors, the design of the wearable device, the design and debugging of the circuitry, which are closely linked and we agree that there is no need for an overly clear distribution of work, Rui Cai will lead the development and fabrication of the circuitry. Yingzhuo Wang (ME) will be responsible for the development of the wireless Bluetooth data transmission technology, the visualization of the monitoring results and the implementation of the interactive functions.