Professor Sheng bin of Jiaotong University: in the metaverse boom, we should pay more attention to the breakthrough of research and application transformation mode

Surging journalist Shao Wen

A6021 figure: Qian Xuesen translated VR into “spiritual realm” in his letter

“It is a good thing that people from all walks of life at home and abroad warmly discuss the concepts of virtual reality and metaverse, and it is also a catalyst to promote the progress of relevant disciplines. However, as for scientific research, I think the most important thing may not be the progress of a technology or algorithm itself, but the breakthrough of relevant technology to the thinking paradigm of traditional scientific research and the change of human life, sports and health management in the past.” Sheng bin, Professor of computer science and Engineering Department of Shanghai Jiaotong University, said in an exclusive interview with surging news.

Sheng bin is a young top talent winner of the national “ten thousand talents program”, the head of intelligent medical technology of the Key Laboratory of artificial intelligence of the Ministry of education and Shanghai Institute of artificial intelligence. He has done research in the fields of artificial intelligence and virtual reality (VR). Based on the world’s largest community fundus image database, led by academician Jia Weiping of the Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University, the Sheng Bin research group has constructed an intelligent diagnosis system for the whole course of diabetic retinopathy, the DeepDR system. It can give real-time feedback on the quality of fundus image and recognize and segment fundus lesions. The paper was selected as editors’ highlights by nature communications, a subsidiary of nature, and is considered to be “one of the most important technological innovations and research achievements in the field of transformation and clinical research” in the international academic community recently.

In the field of virtual reality, from the in-depth participation of Sheng Bin’s team in the key project of “science and technology Winter Olympics” led by Shanghai Institute of physical education in 2019, a snowmobile simulation training platform based on virtual reality has been built for Chinese athletes preparing for the 2022 Beijing Winter Olympics. Due to the high cost and high technical requirements of snowmobile slideways, 17 snowmobile slideways are distributed in a very few countries all over the world. The extreme demand for relevant venue resources greatly puzzles Chinese athletes in preparing for the competition. For example, due to the lack of appropriate track support, the first China snowmobile championship in 2017 was held in Calgary, Canada.

   “With the strong support and help of Professor Wang Ran and Professor Gao Binghong of Shanghai Institute of physical education, we have solicited the opinions and feedback of Chinese national team leaders, coaches and athletes for many times. Our purpose is to face the technical challenge that puzzles the large-scale popularization of ice and snow sports in China, and then fundamentally promote the improvement of China’s ice and snow sports level through scientific and technological forces. With the support of scientific and technological Winter Olympics, we Shape modeling and rendering, physical simulation in ice and snow sliding, and highly immersive virtual reality interaction will transform the snowmobile training in the virtual environment into a virtual simulation assisted teaching and training system with high-level competitive experience effect, rather than games. ” Sheng Bin said.

“Ice and snow sports should not only belong to Gu ailing,” Sheng Bin said with a smile. “Virtual reality technology can let more Chinese teenagers experience the happiness brought by ice and snow sports.” Since 2019, the snow and ice sports virtual simulation system and its teaching application developed by Sheng Bin’s team have been selected as the Shanghai virtual simulation teaching project and the school level first-class course of Shanghai Jiaotong University by Shanghai Municipal Education Commission.

In January 2022, on the eve of the Beijing Winter Olympic Games, led by Vice President Wang Jihong of Shanghai Institute of physical education, industry, University, research and application units such as Shanghai Jiaotong University, Baoshan Education Bureau and Shanghai Institute of artificial intelligence jointly established the “application practice community of the Ministry of Intelligent Physical Education” with the support of the Ministry of education. One of its tasks is to rely on the snow and ice sports virtual simulation system developed by Sheng Bin’s team, It will be extended to the physical education teaching and scientific and technological experience of primary and secondary schools in Baoshan District, Shanghai, to help more Shanghai teenagers understand and love ice and snow sports. Sheng Bin said, “it’s equivalent to leaving a little flame in his heart.”

“Our country’s competitive performance in the Winter Olympics has not been outstanding for a long time. The first reason is that there are not enough ice and snow geographical resources. The second reason is that there are shortcomings in our knowledge popularization of ice and snow sports for teenagers.” Sheng Bin said. Not only in China, in fact, except for those countries that have hosted the Winter Olympic Games, few countries will build this kind of professional slideway with high investment and low output – large ski resorts can also be converted into civilian leisure facilities after the competition. A purely professional track is really difficult to have the value of follow-up utilization other than training and competition. Moreover, the speed of snowmobile is generally up to 130 kilometers per hour, and the world record has exceeded 200 kilometers per hour. It is known as “F1 on snow”. For safety reasons, it also has high professional requirements for participants.

In this regard, Sheng bin believes that, “Whether it’s steel snowmobile, skiing or speed skating, the success of these athletes is due to their physical conditions, interests and hobbies. However, China has a large population and has rich physical fitness and sports talent resources suitable for ice and snow sports. However, due to geographical and climatic factors, not all of them can easily get in touch with ice and snow sports in the enlightenment stage, and a large number of ice and snow talents may be buried. That’s why Well, the ice and snow sports simulation system in the virtual reality environment jointly developed by Shanghai Jiaotong University and Shanghai Institute of physical education may help China’s future talent reserve of ice and snow sports to the greatest extent and help them take the first step in their ice and snow life. “

Sheng bin also further explained the technical background behind the virtual simulation system of ice and snow sports in combination with the current hot concept of “metaverse”, “Our ice and snow sports virtual simulation system can be regarded as the metaverse application of ice and snow sports teaching, and metaverse can be regarded as the expression form of human network life in the virtual reality environment. The so-called virtual in virtual reality refers to the use of computer technology as the core and combined with relevant scientific fields to generate a virtual digital environment to provide people’s vision, hearing, touch, spatial information and other perception The so-called reality refers to such a digital virtual environment and all kinds of perceptual information it provides to people, which can immerse people and produce a sense of reality. “

VR initial maturity: vertigo has not been completely solved

In 5g and even the post 5g era, the basic hardware supporting the large-scale development of VR, such as chips, sensors, optical elements, display screens, etc., have mature development conditions. Roblox, a metaverse concept stock company that uses VR and other devices as interfaces, has a market value of US $50 billion after its listing in March 2021. The acquisition of Pico, the first VR manufacturer in China, by the latter has once again attracted widespread attention. In addition, Goethe group began to lay out the VR / AR industry in 2012. At present, Goethe group occupies 70% of the market share of the world’s medium and high-end virtual reality head display, and takes the lead in introducing the world’s leading 12 inch nano embossing production line to build a world leading VR / ar high-end optical component industry base.

According to IDC’s report, the number of VR devices will increase from about 5 million in November 2020. Meta (former Facebook) CEO Zuckerberg once said that about 10 million people need to use and buy VR content on a platform to enable developers to continue research and development and make profits; Once this threshold is exceeded, the content and ecosystem will achieve leapfrog development.

In the construction of domestic academic ecology of virtual reality, the international journal virtual reality & amp; Less than three years after its publication, intelligent hardware (vrih) was included by EI Compendex, the world’s most extensive and complete engineering literature database, at the end of January this year. In October this year, it will jointly organize a special edition of “metaverse” with computer graphics International (cgi2022), the oldest well-known international academic conference in the field of virtual reality and graphics. As the editorial board of vrih, Sheng bin believes that, The above events have proved that the academic and industrial achievements of Chinese virtual reality have increasingly important influence in the world.

In the process of designing the virtual simulation system of ice and snow sports, Sheng bin realized that the most subtle thing is to cross the limitations of the inherent knowledge field, carry out realistic modeling and Simulation for a sport that has no previous experience, and is committed to using virtual reality and artificial intelligence technology to restore and reproduce the real experience and feelings of others on this activity to the greatest extent. “This is actually one of the charms of metaverse. To really do this well, in addition to tackling difficult problems in modeling, rendering or interaction technology, it is also important to integrate other disciplines, such as sports physiology and the movement law of some equipment on ice, into the process of system development.”

According to the “white paper 2021 on the development of virtual reality industry” released by CCID Research Institute, there is no breakthrough in the key core technologies such as vertigo and interaction in the current virtual reality industry, and there is still a gap between the wearing comfort, visual comfort and comfort experience of VR, AR, Mr and other terminal devices.

For dizziness, Sheng bin explained, “In essence, dizziness actually comes from the discomfort caused by the brain when processing ‘unreal information’. For example, the unreal feeling caused by visual perception, the flickering and low resolution of the picture, or the abnormal light and shadow effect will bring discomfort. There are many technologies to deal with dizziness, but I don’t think this problem has been completely overcome. That’s why many people talk about metaverse now, but the actual experience will be different Stuck on device. “

“Ultimate display”: the virtual reality window of human digital society in the future – metaverse

As early as 1965, Ivan Sutherland, the father of computer graphics, published a short article entitled the ultimate display, which proposed that the ultimate goal of display technology is to make people directly immersed in the computer-controlled virtual environment, just like living in the real world. “At that time, the expectation of the ultimate state of this display technology was still the window of windows system. People could walk into windows, just like Alice in Wonderland, but they could move, listen, see, touch and smell.” Sheng Bin said.

This idea of “ultimate display” continues to today, and there is the saying of “next generation general computing platform”. In 2016, Goldman Sachs released the report “VR and AR: interpreting the next general computing platform” (ar Augmented Reality). In February 2022, Guotai Junan released the report MR: the next generation general computing platform leading to metaverse (MR mixed reality).

Sheng bin revealed that he is currently working closely with Academician Jia Weiping, a chair professor of Shanghai Jiaotong University, to carry out scientific research with a multidisciplinary expert team from sports science, clinical medicine, psychology and brain science, and to develop a “spiritual” system for the intelligent and fine management of people’s health in the future, Through artificial intelligence and virtual reality, I hope to answer the problem that has plagued the scientific community for many years: what is the impact of real-world physical exercise and metaverse VR on human physical and mental health.

Sheng bin believes that behind these efforts lies his scientific research belief. “If we finally communicate with machines through AR, VR or MR, virtual simulation and artificial intelligence can fully realize the simulation and prediction of the change law of the real world and human body state through the comprehensive use of interdisciplinary knowledge and based on big data and high-performance general computing platform.”

图:钱学森在书信中表达关于VR的未来期待 figure: Qian Xuesen expressed his expectations for the future of VR in his letter

Leave a comment

Cancel reply

Your email address will not be made public. Required items have been marked with *


Display name*

Email address*

Website address

Save my display name, email address and website address in this browser for the next comment.