Computer Vision Decoded – Détails, épisodes et analyse

In this episode, Jonathan Stephens and Jared Heinly delve into the intricacies of COLMAP, a powerful tool for 3D reconstruction from images. They discuss the workflow of COLMAP, including feature extraction, correspondence search, incremental reconstruction, and the importance of camera models. The conversation also covers advanced topics like geometric verification, bundle adjustment, and the newer GLOMAP method, which offers a faster alternative to traditional reconstruction techniques. Listeners are encouraged to experiment with COLMAP and learn through hands-on experience.

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode, we discuss practical tips and challenges in 3D reconstruction from images, focusing on various environments such as urban, indoor, and outdoor settings. We explore issues like repetitive structures, lighting conditions, and the impact of reflections and shadows on reconstruction quality. The conversation also touches on the importance of camera motion, lens distortion, and the role of machine learning in enhancing reconstruction processes. Listeners gain insights into optimizing their 3D capture techniques for better results.

Key Takeaways

• Repetitive structures can confuse computer vision algorithms.
• Lighting conditions greatly affect image quality and reconstruction accuracy.
• Wide-angle lenses can help capture more unique features.
• Indoor environments present unique challenges like textureless walls.
• Aerial imaging requires careful management of lens distortion.
• Understanding the application context is crucial for effective 3D reconstruction.
• Camera motion should be varied to avoid distortion and drift.
• Planning captures based on goals can lead to better results.

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services. Learn more at https://www.everypoint.io

In this episode of Computer Vision Decoded, we are going to dive into image capture best practices for 3D reconstruction.

At the end of this livestream, you will have learned the basics for capturing scenes and objects. We will also provide a downloadable visual guide for reference on your next 3D reconstruction project.

Download the official guide here to follow along: https://tinyurl.com/4n2wspkn

00:00 Intro
04:40 Camera motion overview
07:15 Good camera motions
18:43 Transition camera motions
30:39 Bad camera motions
39:27 How to combine camera motions
49:16 Loop Closure
57:42 Image Overlap
1:14:00 Lighting and camera gear

Watch out episode of Computer Vision in the Wild to learn more about capturing images outside and in busy locations: https://youtu.be/FwVBR6KFjPI

Follow Jared Heinly on Twitter: https://twitter.com/JaredHeinly
Follow Jonathan Stephens on Twitter at: https://twitter.com/jonstephens85

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded, we join Jared Heinly and Jonathan Stephens from EveryPoint for their live reaction to the iPhone 14 series announcement. They go in depth into what all the camera specs mean to the average person. We also explain basics of computational photography and how Apple is able to get great photos from a small camera sensor.

00:00 Intro
02:43 Apple Watch Review
06:58 Airpods Pro Review
09:40 iPhone 14 Initial Reaction
15:05 iPhone 14 Camera Specs Breakdown
37:13 iPhone 14 Pro Initial Reaction
40:47 iPhone 14 Pro Camera Specs Breakdown

Follow Jared Heinly on Twitter
Follow Jonathan Stephens on Twitter

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded, we sit down with Jared Heinly, Chief Scientist at EveryPoint, to discuss 3D reconstruction in the wild. What does “in the wild” mean? This means 3D reconstructing objects and scenes in non-controlled environments where you may have limitations with lighting, access, reflective surfaces, etc.

00:00 Intro
01:30: What are Duplicate Scene Structures and How to Avoid Them
14:30: How Jared used 100 million crowdsourced photos to 3d reconstruct 12,903 landmarks
27:10: The benefits of capturing video for 3D reconstruction
31:30: The benefits of using a drone to capture stills for 3D reconstruction
34:20: Considerations for using installed cameras for 3d reconstruction
38:30: How to work with sun issues
44:25: Determining how far from the object you should be when capturing images
50:35: How to capture objects with reflective surfaces
53:40: How work around scene obstructions
57:20: What cameras you should use

Jared Heinly’s Academic Papers and Projects

Paper: Correcting the Duplicate Scene Structure In Sparse 3D Reconstruction
Project: Reconstructing the World in Six Days
Video: Reconstructing the world in Six Days

Follow Jared Heinly on Twitter
Follow Jonathan Stephens on Twitter

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded we dive into Jared Heinly's recent trip to the CVPR Conference. We cover: what the conference about, who should attend, what are the emerging trends in computer vision, how machine learning is being used in 3D reconstruction, and what NeRFs are for.

00:00 - Introduction
00:36 - What is CVPR?
02:49 - Who should attend CVPR?
08:11 - What are emerging trends in Computer Vision?
14:34 - What is the value of NeRFs?
20:55 - How should you attend as a non-scientist or academic?

Follow Jared Heinly on Twitter
Follow Jonathan Stephens on Twitter

CVPR Conference

Episode sponsored by: EveryPoint

In this inaugural episode of Computer Vision Decoded we dive into the recent announcements at WWDC 2022 and find out what they mean for the computer vision community. We talk about what Apple is doing with their new RoomPlan API and how computer vision scientists can leverage it for better experiences. We also cover the enhancements to video and photo capture during an active ARKit Session.

00:00 - Introduction
00:25 - Meet Jared Heinly
02:10 - RoomPlan API
06:23 - Higher Resolution Video with ARKit
09:17 - The importance of pixel size and density
13:13 - Copy and Paste Objects from Photos
16:47 - CVPR Conference Overview

Follow Jared Heinly on Twitter
Follow Jonathan Stephens on Twitter

Learn about RoomPlan API Overview
Learn about ARKit 6 Highlights
CVPR Conference

Episode sponsored by: EveryPoint

In this episode of Computer Vision Decoded, Jonathan Stephens and Jared Heinly explore the concept of depth maps in computer vision. They discuss the basics of depth and depth maps, their applications in smartphones, and the various types of depth maps. The conversation delves into the role of depth maps in photogrammetry and 3D reconstruction, as well as future trends in depth sensing and machine learning. The episode highlights the importance of depth maps in enhancing photography, gaming, and autonomous systems.

Key Takeaways:

• Depth maps represent how far away objects are from a sensor.
• Smartphones use depth maps for features like portrait mode.
• There are multiple types of depth maps, including absolute and relative.
• Depth maps are essential in photogrammetry for creating 3D models.
• Machine learning is increasingly used for depth estimation.
• Depth maps can be generated from various sensors, including LiDAR.
• The resolution and baseline of cameras affect depth perception.
• Depth maps are used in gaming for rendering and performance optimization.
• Sensor fusion combines data from multiple sources for better accuracy.
• The future of depth sensing will likely involve more machine learning applications.

Episode Chapters
00:00 Introduction to Depth Maps

00:13 Understanding Depth in Computer Vision

06:52 Applications of Depth Maps in Photography

07:53 Types of Depth Maps Created by Smartphones

08:31 Depth Measurement Techniques

16:00 Machine Learning and Depth Estimation

19:18 Absolute vs Relative Depth Maps

23:14 Disparity Maps and Depth Ordering

26:53 Depth Maps in Graphics and Gaming

31:24 Depth Maps in Photogrammetry

34:12 Utilizing Depth Maps in 3D Reconstruction

37:51 Sensor Fusion and SLAM Technologies

41:31 Future Trends in Depth Sensing

46:37 Innovations in Computational Photography

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services. Learn more at https://www.everypoint.io

After an 18 month hiatus, we are back! In this episode of Computer Vision Decoded, hosts Jonathan Stephens and Jared Heinly discuss the latest advancements in computer vision technology, personal updates, and insights from the industry. They explore topics such as real-time 3D reconstruction, computer vision research, SLAM, event cameras, and the impact of generative AI on robotics. The conversation highlights the importance of merging traditional techniques with modern machine learning approaches to solve real-world problems effectively.

Chapters

00:00 Intro & Personal Updates
04:36 Real-Time 3D Reconstruction on iPhones
09:40 Advancements in SfM
14:56 Event Cameras
17:39 Neural Networks in 3D Reconstruction
26:30 SLAM and Machine Learning Innovation
29:48 Applications of SLAM in Robotics
34:19 NVIDIA's Cosmos and Physical AI
40:18 Generative AI for Real-World Applications
43:50 The Future of Gaussian Splatting and 3D Reconstruction

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io

In this episode of Computer Vision Decoded, we are going to dive into our in-house computer vision expert's reaction to the iPhone 15 and iPhone 15 Pro announcement.

We dive into the camera upgrades, decode what a quad sensor means, and even talk about the importance of depth maps.

Episode timeline:

00:00 Intro
02:59 iPhone 15 Overview
05:15 iPhone 15 Main Camera
07:20 Quad Pixel Sensor Explained
15:45 Depth Maps Explained
22:57 iPhone 15 Pro Overview
27:01 iPhone 15 Pro Cameras
32:20 Spatial Video
36:00 A17 Pro Chipset

This episode is brought to you by EveryPoint. Learn more about how EveryPoint is building an infinitely scalable data collection and processing platform for the next generation of spatial computing applications and services: https://www.everypoint.io