Raymond Lo 👓🤖

The automation process of connecting the real-world is one of the key building blocks for creating a truly immersive metaverse. With the advance of deep learning techniques, we could expect that in future of AR/VR, OCR along with natural language processing (NLP), could help us to digitize, structure or translate all text information according to our needs, from different sources to every AR, VR or mobile wearable device. With the mapping of real-world information to the virtual world, existing… Show more

The automation process of connecting the real-world is one of the key building blocks for creating a truly immersive metaverse. With the advance of deep learning techniques, we could expect that in future of AR/VR, OCR along with natural language processing (NLP), could help us to digitize, structure or translate all text information according to our needs, from different sources to every AR, VR or mobile wearable device. With the mapping of real-world information to the virtual world, existing or historical knowledge could be best utilized to generate more creative and resourceful results for a more splendid metaverse.
Optical character recognition (OCR) is one of the earliest addressed computer vision tasks, for which different implementations have been proposed. Although in some specific use cases, certain OCR implementations yield very-good results, up to 99% accuracy, a general OCR that could provide high accuracy in different text density, structures of text, fonts, artifacts or text locations is still quite challenging. Therefore, deep-learning based OCR implementation has been widely investigated in recent years, given its strong learning abilities and high robustness. In this hands-on course, what we are learning is 1) a working pipeline to perform automatic digitization based on OCR and NLP algorithms, for uploaded images or through an AR demo; 2) a step-by-step tutorial on how to use OpenVINO to accelerate the inference for OCR and NLP; 3) evaluation and deployment of the solution as an edge-computing system with plain CPU. Show less

https://paularamo.github.io/cvpr-2022/

How to get quick and performant model for your edge application. From data to application.
Organizers:
Paula Ramos, Raymond Lo, Helena Kloosterman, Samet Akcay, Zhuo Wu & Yury Gorbachev

In this thesis, I present a new generation of Augmediated Reality Digital Eye Glass (DEG) that embodies the principles of Humanistic Intelligence (HI). In particular, this thesis addresses the fundamental issues in the practical realization of the HI framework, most notably the dynamic range of sensors and the design of a natural human-computer interface based on three dimensional (3D) gestural input. First, to address the dynamic range limitation of color sensors, a practical implementation of… Show more

In this thesis, I present a new generation of Augmediated Reality Digital Eye Glass (DEG) that embodies the principles of Humanistic Intelligence (HI). In particular, this thesis addresses the fundamental issues in the practical realization of the HI framework, most notably the dynamic range of sensors and the design of a natural human-computer interface based on three dimensional (3D) gestural input. First, to address the dynamic range limitation of color sensors, a practical implementation of high dynamic range (HDR) image composition and tone mapping algorithms were designed on both Field-programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs) to enable real-time processing of video streams. The algorithms are optimized for wearable applications and provide a constant runtime performance independent of the scene. To demonstrate the robustness of the algorithm, a specialized DEG helmet was designed to work under the most extreme lighting condition - tungsten inert gas welding. In addition, I discuss the development of a novel HDR technique for 3D depth sensing cameras called 3DHDR. Here, I present a method of creating 3D HDR depth maps using an array of 3D depth sensing cameras designed for wearable applications. The system provides a tonal and spatial range that significantly exceeds the capability of any individual depth sensor. Finally, I present an untethered wearable solution of DEG with 3D gestural input based on 3DHDR. The recognition system provides various mode of operations and allows users to integrate gesture-based input to control the Augmediated Reality DEG system. The final prototype has been translated to a development platform for use in future wearable computing research to further explore the HI framework. Show less

Overlaying and registering interactive virtual content on the real world requires sophisticated hardware and advanced software: cutting-edge optics, state-of-the-art sensors, and the latest in computer vision and computer graphics algorithms. Meta 2 provides an easy-to-use platform for creating augmented reality applications that abstracts these complexities.

OpenGL is a great multi-platform, cross-language, and hardware-accelerated graphics interface for visualizing large 2D and 3D datasets. Data visualization has become increasingly challenging using conventional approaches as datasets become larger and larger, especially with the Big Data evolution. From a mobile device to a sophisticated high-performance computing cluster, OpenGL libraries provide developers with an easy-to-use interface to create stunning visuals in 3D in real time for… Show more

OpenGL is a great multi-platform, cross-language, and hardware-accelerated graphics interface for visualizing large 2D and 3D datasets. Data visualization has become increasingly challenging using conventional approaches as datasets become larger and larger, especially with the Big Data evolution. From a mobile device to a sophisticated high-performance computing cluster, OpenGL libraries provide developers with an easy-to-use interface to create stunning visuals in 3D in real time for a wide range of interactive applications.

This book provides an easy-to-follow, hands-on tutorials to create appealing OpenGL-based visualization tools with minimal development time. We will first illustrate how to quickly set up the development environment in Windows, Mac, and Linux. Next, we will demonstrate how to visualize data for a wide range of applications using OpenGL, starting from simple 2D datasets to increasingly complex 3D datasets with more advanced techniques. Each chapter addresses different visualization problems encountered in real life and introduces the relevant OpenGL features and libraries in a modular fashion. By the end of this book, you will be equipped with the essential skills to develop a wide range of impressive OpenGL-based applications for your unique data visualization needs, on platforms ranging from conventional computers to the latest mobile/wearable devices.
Show less