NOTE: This group will have its meetings during working hours (9am-5pm timeframe on weekdays) and in-person attendance is required.
This project is for students with the following interests:
The project this year is not an IC design project. This is a hands-on hardware/software design that will be built and in operation by the end of the project.
Ok, so what will actually be done in the project? Here’s the nuts and bolts: we need to design a device that can capture the 3D geometry of small items. We’ll do that using something called photogrammetry, and possibly also using something called structured lighting. We’ll design a mechanical device that spins a target item around so we can taken dozens of pictures of it from all angles. Or, we’ll design a mechanical device that moves a camera around an target item, and takes dozens of pictures of it from all angles. Either way works.
Once we have all the pictures, it’s possible to reconstruct the 3D geometry of the target item using the photos. This is the photogrammetry part. Now, the mathematics are involved, and teams of people write such software, so we will not be writing the photogrammetry software. We’ll use existing software and interface to it through its API.
From our project viewpoint we need to:
This is a high level description and the detailed design will require a lot of engineering design work on the hardware and software. There is more than enough workload here for a team of 3-5 students, depending on the students’ skill sets.
My “napkin sketch” of the system is below. We might take things a different way, so this is just one possibility.
NVIDIA has released the source code to their NeuroAngelo 3D reconstruction software. It combines the concepts of photogrammetry with neural networks.
https://github.com/NVlabs/neuralangelo
It relies upon another bit of software called COLMAP, which is open-source photogrammetry software. I plan to combine NeuroAngelo and COLMAP with custom mechatronics to create a super-fast object-to-3D model pipeline.