Expert consensus on the metaverse in medicine
Dawei Yang a,m,n , Jian Zhou a,m,n , Rongchang Chen b , Yuanlin Song a,m,n , Zhenju Song c,n,
Xiaoju Zhang d , Qi Wang e , Kai Wang f , Chengzhi Zhou g , Jiayuan Sun h , Lichuan Zhang i , Li Bai j,
Yuehong Wang k , Xu Wang l , Yeting Lu m, Hongyi Xin o , Charles A. Powell p , Christoph Thüemmler q,
Niels H. Chavannes r , Wei Chen s,t , Lian Wu u , Chunxue Bai a,m,n,⇑
aDepartment of Pulmonary and Critical Care Medicine, Zhongshan Hospital Fudan University, Shanghai, China
b Shenzhen Institute of Respiratory Disease, First Affifiliated Hospital of South University of Science and Technology of China (Shenzhen People’s Hospital), Shenzhen, Guangdong, China
cDepartment of Emergency Medicine, Zhongshan Hospital Fudan University, Shanghai, China
dDepartment of Respiratory and Critical Care Medicine, Henan Provincial People’s Hospital, Zhengzhou, China
eDepartment of Respiratory Medicine, The Second Hospital, Dalian Medical University, Dalian, China
f Department of Pulmonary Medicine, The Second Affifiliated Hospital, Zhejiang University, Hangzhou, China
gDepartment of Thoracic Surgery and Oncology, The First Affifiliated Hospital of Guangzhou Medical University, National Center for Respiratory Medicine, State Key Laboratory of
Respiratory Disease and National Clinical Research Center for Respiratory Disease, Guangzhou, China
hDepartment of Respiratory Endoscopy, Department of Respiratory and Critical Care Medicine, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, China
i Zhongshan Hospital Affifiliated of Dalian University, Dalian, Liaoning, China
j Department of Respiratory Critical Care Medicine, Xinqiao Hospital, Army Medical University (Third Military Medical University), Chongqing, China
kDepartment of Respiratory Disease, The First Affifiliated Hospital, College of Medicine, Zhejiang University, Hangzhou, China
l The Affifiliated Wuxi No.2 People’s Hospital of Nanjing Medical University, Wuxi, Jiangsu, China
m Shanghai Engineer & Technology Research Center of Internet of Things for Respiratory Medicine, Shanghai, China
n Shanghai Key Laboratory of Lung Inflflammation and Injury, Science and Technology Commission of Shanghai Municipality
oUM-SJTU Joint Institute, Shanghai Jiaotong University, Shanghai, China
p Pulmonary, Critical Care and Sleep Medicine, Icahn School of Medicine at Mount, New York, NY, USA
q Helios Park-Klinikum Leipzig, Leipzig, Germany
r Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
sDivision of Pulmonary Medicine, Children’s Hospital of Pittsburgh of UPMC, Pittsburgh, PA, USA
t Department of Pediatrics, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
u School of Healthcare and Social Practice, Unitec Institute of Technology, Auckland, New Zealand
article info
Article history:
Available online 4 February 2022
Keywords:
Medical Internet of Things (MIoT)
Metaverse
Metaverse in medicine
Virtuality-reality integration
Virtuality-reality interconnection
abstract
Background: Recently, Professor Chunxue Bai and colleagues have proposed a defifinition of the Metaverse
in Medicine as the medical Internet of Things (MIoT) facilitated using AR and/or VR glasses.
Methods: A multi-disciplinary panel of doctors and IT experts from Asia, the United States, and Europe
analyzed published articles regarding expert consensus on the Medical Internet of Things, with reference
to study results in the fifield of metaverse technology.
Findings: It is feasible to implement the three basic functions of the MIoT, namely, comprehensive per
ception, reliable transmission, and intelligent processing, by applying a metaverse platform, which is
composed of AR and VR glasses and the MIoT system, and integrated with the technologies of holographic
construction, holographic emulation, virtuality-reality integration, and virtuality-reality interconnection.
In other words, through interactions between virtual and real cloud experts and terminal doctors, we will
be able to carry out medical education, science popularization, consultation, graded diagnosis and treat
ment, clinical research, and even comprehensive healthcare in the metaverse. The interaction between
virtual and real cloud experts and terminal users (including terminal doctors, patients, and even their
family members) could also facilitate different medical services, such as disease prevention, healthcare,
physical examination, diagnosis and treatment of diseases, rehabilitation, management of chronic dis
eases, in-home care, fifirst aid, outpatient attendance, consultation, etc. In addition, it is noteworthy that
security is a prerequisite for the Metaverse in Medicine, and a reliable security system is the foundation
to ensure the normal operation of such a platform.
https://doi.org/10.1016/j.ceh.2022.02.001
2588-9141/ 2022 The Authors. Publishing services by Elsevier B.V. on behalf of KeAi Communications Co., Ltd.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
⇑ Corresponding author.
E-mail address: cxbai@fudan.edu.cn
(C. Bai).
Clinical eHealth 5 (2022) 1–9
Contents lists available at ScienceDirect
Clinical eHealth
journal homepage: ww.keaipublishing.com/CEHConclusion:
The application of a Cloud Plus Terminal platform could enable interaction between virtual
and real cloud experts and terminal doctors, in order to realize medical education, science popularization,
consultation, graded diagnosis and treatment, clinical research, and even comprehensive healthcare in
the metaverse.
2022 The Authors. Publishing services by Elsevier B.V. on behalf of KeAi Communications Co., Ltd. This
is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/
4.0/).
- Introduction
Since 2021, the concept of the metaverse has been widely dis
cussed. It refers to the internet accessed via virtual reality (VR)
and augmented reality (AR) glasses, and is considered to be the
next-generation mobile computing platform that will be widely
used in the future.1 Others believe that the metaverse is a ternary
digital world established on the basis of digital technology inte
grating the virtual and the real worlds, which people enter with
digital identities. The idea originated from the novel True Names
by Professor Vernor Vinge, an American mathematician. In this
story, published in 1981, the author creatively conceived a virtual
world that enters and obtains sensory experience through a brain
computer interface. Later, in 1992, the term ‘‘metaverse” was
coined by American science-fifiction writer Neal Stephenson in his
novel Snow Crash, in which the characters explore an internet
world parallel to the real world, using digital avatars of themselves
for perception and interaction.2–3
The rise of the metaverse has brought infifinite possibilities to all
types of sectors and occupations, such as video game production,
leisure, and entertainment. Museum exhibitions have evolved with
diverse digital technologies,4 and new metaverse sales models
have emerged from traditional retail.5 Some researchers have stud
ied the art community of the 3D virtual world. Concerning social
media, on October 28th, 2021, Mark Zuckerberg announced that
Facebook had changed its name to ‘‘Meta” to align the company
with its focus on new computing technologies and the metaverse.6
Others seek to understand how journalism is practiced in the
metaverse.7–8
Recently, Bai and colleagues proposed the metaverse in
medicine,9 and suggested naming 2022 as the Year of the Meta
verse in Medicine. The expert group further discussed the defifini
tion of the metaverse in the medical context, and its concept and
application scenarios, as well as its clinical importance. It is
expected that this new concept will contribute to improving com
prehensive healthcare as well as prevention and treatment of dis
eases, and to upgrading the current diagnosis and treatment model
– which varies between doctors and hospitals, creating uneven
standards akin to production in handicraft workshops (referred
to as the ‘‘handicraft workshop model” hereinafter) – to a modern
assembly-line model that meets national and even international
standards.
- The concept of a metaverse and its possible applications in
medicine
To understand the validity and feasibility of applying the meta
verse in medicine, it is necessary to fifirst understand its concept.
The metaverse is the internet accessed via VR and AR glasses,1
which has been increasingly acknowledged, and is considered as
a manifestation of next-generation mobile computing platforms.
Similarly, the Metaverse in Medicine can be defifined as the medical
Internet of Things (MIoT) facilitated using AR and VR glasses.
The current practice also implies the wide scope of applica
tions of the metaverse, including general settings, such as social
activities, e-commerce, education, gaming, and payments,10 and
special fifields, such as medicine.9 In fact, many of the internet
based applications that we are familiar with already have a pres
ence in the metaverse. Looking back through history, the per
sonal computer (PC), undoubtedly the mainstream computing
platform of the 1990s, was applied in telemedicine.11 Later, we
witnessed the rise of the mobile phone. This has gradually
replaced the PC,12 and has been integrated into the internet or
the Internet of Things (IoT).13
Today, many people believe that VR and AR glasses will become
an important part of next-generation computing platforms. Internet
applications will also undergo updates and iterative development
along with the replacement of computing platforms. For instance,
where previously we had internet-based instant messaging soft
ware on PCs, including QQ and Microsoft Service Network (MSN),
for social exchanges, nowadays we use WeChat on our mobile
phones.14–15,16 Similarly, signifificant changes have happened to e
commerce, as new applications emerge in the smartphone era, such
as smoking consumption intervention.17 By deploying the precise
positioning function of smartphones, Local Life is capable of recom
mending high-quality services within a distance of 3 km from users,
something that was impossible in the PC era. It creates a brand-new
user experience based on the new platform, which can also be
applied for the prevention and control of COVID-19.18
Why is the concept of metaverse so widely accepted, and why has
stress been laid on its application in the medical fifield? Because VR
and AR technologies will enable everyone to use digital avatars for
face-to-face communication in the virtual world. Technological
advances have been transforming e-commerce, and may lead to
changes in the applications for comprehensive healthcare and med
ical services. In order to understand the inflfluence of VR and AR
glasses on internet-based applications, we need to analyze the
essence of the technologies adopted by these glasses, which is, to
display core interactions of the new platform. In the past, the two
dimensional display interface determined that all applications were
based on child windows, whether on PCs or mobile phones. This was
the reason that the Microsoft operating system was named ‘‘Win
dows”. The user interaction was completed through mouse clicks
and drags, whereas VR and AR glasses can provide a three
dimensional interface for display and interaction, enabling us to
become immersed in a virtual world of information. Imagine that a
virtual person in front of us is having a conversation with us, or that
there is a virtual shelf fifilled with all sorts of goods next to us. In this
three-dimensional space, interaction can be through body move
ments, language, gestures, and gaze.19 The three-dimensional inter
face for display and interaction is the fundamental setting in such
applications, and the superstructure will undoubtedly undergo rev
olutionary changes, including drastic expansion of its application
scenarios in medicine and comprehensive healthcare.19
The replacement of the computing platform will lead to
tremendous changes in the entire internet industry, including
hardware, software, operating systems, and even the industry
structure.19 Transformation is also expected to take place in the
medical fifield.9 Similarly, to the revolution from PC to smartphone,
today’s technological advances will result in the rise of new key
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
2players in different areas, including in medicine and the healthcare
industry.
- The medical Internet of Things (MIoT) facilitates important
application scenarios for the metaverse
3.1. The MIoT can assist in the practice of P4 medicine
The term ‘‘the Internet of Things” (IoT), coined by Professor
Kevin Ashton at Massachusetts Institute of Technology (MIT) in
1999,20 originally referred to the application of radio-frequency
identifification (RFID) technology and devices, combined with the
Internet, using agreed communication protocols to intelligently
manage objects.
Since 2008, Bai et al. have been developing an innovative wire
less spirometer integrated with a mobile phone, which was fea
tured in an article published by the American Thoracic Society
(ATS) in ATS NEWS (Vol. 35, No. 7/8).21 Being among the fifirst to
introduce the Medical Internet of Things in China and worldwide,
Bai and his team also built the world’s fifirst MIoT-based home
tele-monitoring and management platform for obstructive sleep
apnea-hypopnea syndrome (OSAHS).22
He has served as the Editor
of Practical Medical Internet of Things23
and Guidelines on Applying
Medical Internet of Things for the Graded Diagnosis and Treatment24
(People’s Medical Publishing House), Medical Internet of Things
(Science Press),25 and together with Christoph Thüemmler, as
author of Health 4.0 (Springer Publishing Company).26 Although
the MIoT is still at an early stage of development, it is showing
great potential, and has already been applied in many medical
fifields. Today, the MIoT has become synonymous with a network
of physical objects across the internet,13 integrated with both hard
ware and software, for the purpose of perception, transmission,
and intelligent processing in a variety of medical application sce
narios. We are witnessing signifificant growth in the application of
the MIoT for clinical purposes along with the diverse extension
of embedded devices that integrate the virtual world (information)
and the real world (objects), enabling us to create a huge health
care market that benefifits the patients.
The MIoT has become an increasingly acknowledged concept in
China and abroad. In a recent systematic mapping study by
Sadoughi et al.,27 articles published between 2000 and 2018 in
major online scientifific databases, including IEEE Xplore, Web of
Science, Scopus, and PubMed, were screened, and a total of 3679
papers related to the IoT in medicine were reviewed, amongst
which 89 papers were fifinally selected based on specifific inclu
sion/exclusion criteria. China, India, and the United States were
shown to be the top countries in knowledge production regarding
the MIoT. In addition, the ambiguity of the terms assigned to the
IoT, namely system, platform, device, tool, etc., and their inter
changeable uses in the literature, suggested that a taxonomic study
was required to investigate the precise defifinitions of these terms.
The papers also demonstrated the extensive inflfluence and recogni
tion that the MIoT has gained.
3.2. The MIoT can help improve the quality of healthcare
In the Strategy for American Innovation (2014),28 IT adoption in
medicine and healthcare was considered as one of the 6 priority
fifields for innovation in the USA. The Asthma Health App (AHA)
was designed to conduct large-scale health research and provide
real-time air pollution monitoring. Based on data analysis of the
users’ electronic asthma diary, this app can predict acute attacks,
contributing to the primary and secondary prevention of the dis
ease.29 As part of the Leading Age Center for Aging Services Tech
nologies (CAST), Intel developed wireless sensor networks
(WSNs) for in-home healthcare solutions.30 Sensing devices
embedded in objects such as shoes, furniture, and home appli
ances, could make it possible for the elderly and those with disabil
ities to continue to live independently at home, while medical staff
and social workers could also provide assistance when necessary.
Sponsored by the Defense Advanced Research Projects Agency
(DARPA), MIT conducted research on ultra-low-power WSNs, while
Auburn University devoted considerable effort to studying self
organizing sensor networks,31 and completed the development of
some experimental systems. Scientists at the University of Roche
ster built a smart medical room equipped with wireless sensors in
which dust was used to measure important signs of the occupant
(such as blood pressure, pulse, and respiration), sleeping position,
and 24-hour daily activities.32
The AMON project,33 funded by the EU IST FP5 program with
the participation of several research institutes, aimed to develop
a wearable tele-monitoring and alert system. The wrist-worn
device integrated a system that included continuous collection
and evaluation of multiple vital parameters, intelligent detection
and management of a medical emergency, and a cellular connec
tion to a medical center. STMicroelectronics and Mayo Clinic
jointly developed an innovative telemedicine platform for the
management of chronic cardiovascular diseases.34 Not only did it
perform long-term monitoring without interfering with the
patient’s everyday activities, but it also provided appropriate treat
ment options based on specifific clinical information and physiolog
ical parameters. A study from the University of Malaga and the
University of Almeria proposed a real-time WSN with a specially
designed pulse oximeter, using software installed on a PC or PDA
to monitor the pulse and peripheral oxygen saturation (SpO2) of
different patients at the same time, achieving great simplicity at
a low cost.35
Japan, with its solid network and technological foundation for
the IoT, has also been increasing investment in the sector of med
ical informatization. For example, Toshiba developed an artifificial
intelligence (AI) system composed of wrist-worn wearable sensors
and a PDA that could monitor and analyze the user’s health, daily
activities, and personal habits.36 By offering reminders and advice
on a healthy diet and regular exercise, tailored to specifific individ
uals, the AI played a key role in making behavioral changes and
reducing the risk of lifestyle-related diseases. Based on the wrist
movement, pulse rate, and electrodermal activity, the software
reached 90% accuracy in detecting the user’s activities, such as eat
ing and taking exercise.
China has been researching the application of the MIoT in clin
ical practice since 2008, including the AI-assisted early diagnosis of
lung cancer. Researchers created a database for the training and
validation of a multimodal deep learning model, and established
a cloud computing system based on a graphics processing unit
(GPU) for parallel processing, with access to electronic medical
records (EMRs) and the picture archiving and communication sys
tem (PACS). By developing PNapp5A, an IoT-assisted application
that adopted a 5-step assessment of pulmonary nodules, they man
aged to enhance the early diagnosis of pulmonary nodules using
big data-driven management technologies.37 The team also took
the initiative to develop the Chinese Expert Consensus on the Diagno
sis and Treatment of Pulmonary Nodules,38–40 and promoted the
MIoT platform in around 900 hospitals where the Chinese Alliance
Against Lung Cancer (CAALC) centers and sub-centers are located.
According to Zhongshan Hospital, Fudan University, a total of
16,417 cases of pulmonary nodules underwent surgical treatment
from 2014 to 2019, amongst which 9980 cases (60.8%) of early
stage lung cancers were reported. The patients’ average age
declined from 63 to 50 over the 6 years.41 Based on his experience
of accurately diagnosing pulmonary nodules smaller than 10 mm
with AI assistance, Professor Chunxue Bai proposed the concept
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
3of a human–computer multidisciplinary team (MDT), aiming for
consultation on the basis of human–computer communication
and interaction. Trials have been carried out with outpatient ser
vices adopting a human–computer MDT, providing comprehensive
diagnosis and treatment plans that combine experts’ suggestions
and AI results. This new approach facilitates the standardization
of early-stage lung cancer screening, diagnosis, and treatment for
diffificult cases with indeterminate pulmonary nodules.40
Another clinical application in China is the AI-assisted diagnosis
and treatment of viral pneumonia. A precisely designed intelligent
system with access to the relevant clinical information and CT
imaging can be used for the screening and management of sus
pected cases and indeterminate cases. For example, a mobile
phone-based tool called nCapp was developed for the diagnosis
and treatment of COVID-19, and was recommended by the ATS.42
Furthermore, China is working on applying the MIoT for the man
agement of chronic diseases, such as chronic obstructive pul
monary disease (COPD) and asthma. By leveraging big data
training and the MIoT embedded with a portable spirometer, it is
feasible to provide accurate and personalized guidance and plans
for the daily life activities of each patient, in order to relieve the
condition, improve their quality of life, and prevent acute exacer
bations of the disease.43
3.3. Importance of the MIoT
China has been facing health resource disparities between
regions and hospitals. Small rural hospitals tend to have scant
access to high-end medical devices (‘‘insuffificient equipment cover
age”), the local doctors have limited technical experience (‘‘insuffifi-
cient technical competence”), and the patients often have poor
recognition of medical care (‘‘insuffificient patient satisfaction”).
Because of these ‘‘Three Defificiencies”, many patients prefer to go
to large hospitals and consult prominent doctors for better diagno
sis and treatment, resulting in diffificulties in registration and hospi
talization, which are referred to as the ‘‘Two Diffificulties”.23–25,44
The inflflux of rural patients into city hospitals also restricts the time
that each expert can spend with each patient, leading to limitations
in distributing services in prevention, healthcare, disease manage
ment, and rehabilitation, which we refer to as the ‘‘Four Limita
tions”. To address these issues, we proposed utilizing the MIoT
and suggested using the three basic functions of the MIoT, compre
hensive perception, reliable transmission, and intelligent process
ing, to assist doctors in clinical practice,23–25 which have been
successfully applied in many cases.
Furthermore, due to the increasing demand for healthcare and
the large number of practitioners involved, it is costly to provide
satisfactory and accessible healthcare services for patients. The
IoT, which combines communication technologies with intelligent
mobile devices, is able to play a crucial role in addressing this issue.
As one of the most frequently deployed innovations in the e-health
sector, the MIoT has been redistributing healthcare services from
medical centers to homes and the workplace.45 Since 2018, knowl
edge production in the MIoT and relevant fifields has signifificantly
increased. Additionally, the COVID-19 pandemic has highlighted
the need for the provision of healthcare services to patients at
home, which is also considered as one of the goals of e-health,
and especially of the MIoT.46 On the one hand, IoT applications
are usually developed to save costs, offer greater accessibility for
patients at home, and encourage patient empowerment, which
serves to promote healthcare and personal well-being. On the
other hand, the Digital Twin model, introduced in 2002 by Grieves
as a new standard for Industry 4.0,47 makes it possible for the inte
gration of VR and AR technologies into the MIoT, and for accelerat
ing the transformation into clinical applications with high
effificiency (IEEE – Digital Twin: Enabling Technologies, Challenges
and Open Research). Nevertheless, as an emerging discipline in
applied science, the MIoT is faced with a series of challenges like
any other new medical technology, especially in terms of medical
supervision, medical insurance, and the digital divide, all of which
need to be validated and addressed by large scale clinical applica
tion and promotion in the near future.48
Bai et al. expect that the MIoT will develop into a school of
thought,9 and become a powerful medical tool, since it has the
potential to realize ‘‘simplifification of complex problems, digitaliza
tion of simple problems, programming of digital problems, and
systematization of programming problems”. The ultimate goal is
to upgrade China’s medicine and healthcare from the current
model, which varies between doctors and hospitals and has uneven
service levels, to a modern assembly-line model that meets
national and even international standards, thus fulfifilling our
vision: wise doctors treat patients before onset of diseases, great
doctors benefifit the general public.
3.4. Limitations of the MIoT
eAccording to the present MIoT theory, it is feasible to over
come the ‘‘Three Insuffificiencies”, ‘‘Two Diffificulties”, and ‘‘Four Lim
itations”, and realize effificient and accurate graded diagnosis and
treatment through the linkage between the doctors in large hospi
tals (cloud experts) and the doctors in small and rural hospitals
(terminal doctors). Continuous research and development of
related technologies will also contribute to improving the graded
diagnosis and treatment.27,37 However, the following issues remain
in clinical practice: (1) The cloud experts are not available to par
ticipate in science popularization and professional lectures as if
they were present at all times and in all settings. (2) The cloud
experts are not available to provide guidance for the terminal doc
tors on the diagnosis and treatment at all times and in all settings.
(3) In clinical trials, the major researchers are not available to
supervise the research and instruct the team at all times and in
all settings. (4) Due to the lack of real-time quality control at all
times and in all settings, non-standard diagnosis and treatment,
the so-called ‘‘handicraft workshop model”, still exists to a consid
erable degree.
The real cause lies in the incompatibility between the service
provided by the cloud experts and the needs in the real world,
the inability of the cloud experts to attend to the general public
at all times and in all settings, and the limitations of the internet
technology itself. Therefore, it is necessary to develop an optimized
digital platform in order to tackle the limitations of the MIoT, espe
cially concerning communication and interaction between the
human and the computer, and the integration and interconnection
between the virtual and the real worlds. It is gratifying that the
emergence of the metaverse has provided a possible solution for
all these problems, which also serves as the foundation for the pro
posal and development of the Metaverse in Medicine.26–27,49–50
- The metaverse provides technical support to maximize the
value of the MIoT
4.1. The prototype of the Metaverse in Medicine implies prospects for
its development
The Metaverse in Medicine, which is defifined as the medical
Internet of Things accessed via AR and/or VR glasses, indicates
the importance of AR and VR technologies. We have conducted
extensive research on the MIoT,50 which serves as the foundation
for establishing the Metaverse in Medicine. For example, we
focused on the research and development of the BRM all-in-one
machine, which can be seen as the prototype of the Metaverse in
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
4Medicine (Fig. 1). More recently, we have initiated a related study
to further explore how to implement the Metaverse in Medicine by
applying holographic construction and emulation, and virtuality
reality integration and interconnection (Fig. 2). In order to put it
into better practice, we suggest to broaden the concept by adding
comprehensive perception to the holographic construction, adding
intelligent processing to the holographic emulation, adding quality
control to the virtuality-reality integration, and adding human–
computer integration to the virtuality-reality interconnection, thus
realizing ‘‘simplifification of complex problems, digitalization of
simple problems, programming of digital problems, and systemati
zation of programming problems”.50 This approach can overcome
the obstacle that internet-based healthcare and telemedicine plat
forms hardly play an active role in county hospitals, especially
those in rural villages and towns. Moreover, it will facilitate graded
diagnosis and treatment, and contribute to transforming the cur
rent handicraft workshop model, which varies between doctors
and hospitals with uneven levels, into a modern assembly-line
model that meets national and even international standards.
4.2. Holographic construction and holographic emulation will further
improve the MIoT
Holographic construction, also known as multi-dimensional or
stereoscopic information, refers to a model incorporating all the
information of a certain system, which has been collected and
compiled from multiple channels, perspectives, and positions.51
The data in the system should include not only specifific information
on the working status of each device, data transmission, and sys
tem interaction, but also data on the factors that inflfluence the
operation of the system, such as the natural and social environ
ment in which the system is located. At present, VR home inspec
tion and shop inspection are applications in the holographic
construction.52–53 Holographic emulation is a new feature that
vastly reduces iteration time when developing holographic appli
cations in Unity. Studies have shown that developers creating
applications for Microsoft HoloLens will immediately benefifit from
being able to prototype, debug, and iterate design directly from the
Unity Editor without getting bogged down by long build and
deploy times.54 Although the current research is not applied to
medicine, our preliminary study suggests that holographic emula
tion is a promising technique for the medical fifield because it can
address the issue of how to enable experts to provide services at
all times and in all settings, which cannot be solved by the MIoT.
How can we fully apply holographic construction and holo
graphic emulation in medical practice? The fifirst step is to under
stand the pathological, pathophysiological, or biochemical
changes caused by different diseases, in order to strictly implement
P4 medicine (predictive, preventive, personalized, and participa
tory). To solve practical problems, we suggest to introduce the con
cept of comprehensive perception in the holographic construction
and emulation of the metaverse, since current studies have con-
fifirmed that it can meet the requirements of the Metaverse in Med
icine. A solid technological foundation has already been laid in
medicine, including the use of a variety of sensors applying
photosensitive, gas sensitive, force sensitive, sound sensitive, and
Fig. 1. Application of the BRM all-in-one machine and progress in R&D of the holographic system for the early diagnosis of pulmonary nodules and lung cancer screening.
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
5radiation sensitive components, biochemical examinations to test
liver and kidney function, electrocardiography (ECG), ultrasound,
computed tomography (CT), positron emission tomography
– computed tomography (PET/CT), the spirometer, and the pulse
oximeter. These technologies enable us to monitor the physiologi
cal, pathophysiological, and biochemical changes in the body at all
times and in all settings (or partly), and create a complete info
graphic of the condition of health, sub-health, or disease. Conse
quently, doctors and patients can enter the metaverse with their
own digital twin, and practice metaverse medicine through
virtuality-reality interconnection. Only when the metaverse cre
ates an immersive experience, in which people cannot distinguish
the virtual world from the real one, will it attract the participation
of patients and doctors.55
The Metaverse in Medicine may also be applied to improve the
effificiency of education and training, since it can address the issues
that the cloud experts are not available to participate in science
popularization and professional lectures, or to provide guidance
for the terminal doctors on diagnosis and treatment as if they were
present at all times and in all settings. For instance, we used the
BRM all-in-one machine adopting holographic emulation technol
ogy to show students the mechanism of cigarette smoking-induced
lung cancer.56 This pioneering pedagogical practice produced sen
sational effects because the students observed in an immersive
way the alveolar damage caused by smoking and its relationship
with the onset of lung cancer (Fig. 3). Furthermore, we can also
train students to quickly master various therapeutic techniques
as if they were present in clinical practice, such as magnetic navi
gation, a diffificult technique to apply in surgeries with respiratory
endoscopy. If holographic emulation is used in teaching and clini
cal practice, it will undoubtedly help us to achieve better results
with less effort.57
The research and practice of the Metaverse in Medicine can be
diffificult, since the body structure, etiology, pathological and patho
physiological changes, as well as the pharmacodynamics in differ
ent patients are extremely complicated. However, based on the
successful cases of applying the concept of the Metaverse in Med
icine in the diagnosis and treatment of pulmonary nodules, we
estimate that solutions and principles can be found through careful
classifification of the diseases and extension of the research on
applying the metaverse in the diagnosis and treatment of diseases
to different categories. In other words, we should develop and fol
low the consensus guidelines to ‘‘simplify complex problems”.50
After working out a solution that combines comprehensive percep
tion with holographic construction, the information required for
the holographic construction will be transmitted to the ‘‘Metaverse
in Medicine Cloud”, in preparation for the next step, holographic
emulation plus intelligent processing, and eventually transforming
the real world through the virtual one. Once all these issues are
settled, we will be able to leverage emulation technology in the vir
tual world to seek optimal solutions for the problems in reality,
and map it to the real world through virtuality-reality integration,
so that the virtual and the real experts provide guidance for med
ical practice in the real world.
4.3. Virtuality-reality integration and interconnection can overcome
the limitations of the MIoT
To maximize the value of the MIoT in solving problems for
patients, it is important to provide high-quality assistance in deal
ing with all kinds of issues in clinical practice. This is exactly the
advantage of the Metaverse in Medicine. In addition to ‘‘digitaliz
ing simple problems”, which is already made possible by using
the MIoT, the Metaverse in Medicine provides intelligent diagnosis
or robotic assistance in treatment (such as in surgeries), enabling
all students to have access to hands-on practice to gain experience,
and successfully bridging the gap between education and prac
tice.58 Our previous studies have shown that it is feasible to mon
itor the pathophysiological parameters of diseases with MIoT
connected sensors, and assist clinical diagnosis and treatment
based on intelligent processing of the data. For example, CT images
for early-stage lung cancer screening have been transmitted to a
cloud computer for intelligent processing to obtain assistance in
diagnosis and differential diagnosis.59 The same approach can be
adopted in the Metaverse in Medicine because it is the MIoT facil
itated using AR glasses.
Fig. 2. Flowchart of implementing the Metaverse in Medicine by applying holographic construction and emulation, and virtuality-reality integration and interconnection.
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
6The incorporation of AR glasses into the MIoT takes it to the
next level, virtuality-reality integration. The cloud experts in both
the virtual and the real worlds will guide the terminal doctors at all
times and in all settings, transforming the diagnosis and treatment
from the current handicraft workshop model to a modern
assembly-line model characterized by homogenous service levels
that meet national and even international standards. The
virtuality-reality integration in the Metaverse in Medicine will
effectively strengthen the linkage between the participants (doc
tors and patients), the real environment (devices) and the virtual
environment (virtual doctors, patients, and devices).60 The ulti
mate goal is to create a natural, immersive environment, and inte
grate virtuality into reality to provide medical services based on
human–computer linkage.61
High precision positioning,
virtuality-reality integrated environment presentation, optical dis
plays, and multi-sensory interaction are some of the key technolo
gies required to achieve virtuality-reality integration.62
Additionally, the diverse functions of the MIoT, such as intelligent
diagnosis and treatment, disease management, and especially
quality control, should also be shown on AR glasses. Therefore, it
is crucial to design and produce high-end devices, make elaborate
plans, organize education and trainings, and develop innovative
techniques for quality control.
In an effort to put the Metaverse in Medicine into full practice,
not only should the MIoT devices be taken into consideration, but
the practitioners at grassroots level and specialist doctors should
also be acquainted with the relevant knowledge and skills, and
close collaboration between the cloud experts, the terminal doc
tors, and the patients will be required throughout the practice. In
addition to general trainings, quality control in compliance with
international standards is essential in the clinical application of
the metaverse. (The App developed based on the Metaverse in
Medicine will provide assistance in the quality control of the
MIoT.) For example, in the assessment of pulmonary nodules, inte
gration of experts (in reality) and robots (in virtuality, equipped
with an AI system) ensured that the diagnostic results reached
high sensitivity and specifificity. In order to conduct strict quality
control, we should train the robots with deep learning, and incor
porate the consensus guidelines.63 Currently, quality control can
not be carried out automatically at all times and in all settings,
while the Metaverse in Medicine can overcome these disadvan
tages through virtuality-reality interconnection and human–com
puter integration between the humanoid robots and the cloud
experts, achieving better results with less effort in quality control.
The combination between virtuality-reality interconnection
and human–computer integration is the most important and valu
able feature of the Metaverse in Medicine in clinical practice. In
fact, technologies for virtuality-reality interconnection are quite
mature, but excellent diagnosis and treatment results are not pos
sible without human–computer integration. In theory, human–
computer integration refers to a new form of intelligence gener
ated by the interaction of man, computer, and system environ
ment. In contrast to human intelligence and artifificial intelligence,
the new generation of intelligent system has both physical and bio
logical properties.64 Human-computer interaction mainly involves
physiological and psychological sides of ergonomics that are not
dominated by the brain, while human–computer integrated intel
ligence focuses on the intelligence dominated by the brain com
bined with the ‘‘computer”. From a medical perspective, this
combination, or ‘‘human-computer integration”, refers to the joint
effort of the cloud experts and the robots in communicating with
each other to solve medical problems. We suggested to combine
virtuality-reality interconnection and human–computer integra
tion by programming, in order to ‘‘systematize the problems”,
and proposed the concept of ‘‘human–computer MDT”,50 adopting
programmed digital technology to facilitate virtuality-reality inter
connection. The study results of our clinical application over the
3 years have indicated that human–computer MDT is a perfect
manifestation of the clinical value of the Metaverse in Medicine,
since it signifificantly improves the sensitivity and specifificity in pul
monary nodule assessment. It is believed that this approach can
also be adopted in other application scenarios, such as disease pre
vention, healthcare, self-care, and geriatric nursing, so that the vir
tual and the real cloud experts provide guidance for the terminal
doctors to implement diagnosis and treatment in line with the con
sensus guidelines. For example, during robotic surgery, the cloud
experts can guide a distant robot to perform surgical treatment
on the patient.
- Prospects
This study shows that conditions are mature for the establish
ment of the Metaverse in Medicine, and the experts reached a con
sensus on how to develop it to better serve medicine and
comprehensive healthcare. By applying the Cloud Plus Terminal
Fig. 3. The BRM all-in-one machine adopting holographic emulation technology vividly demonstrates the mechanism of cigarette smoking-induced lung cancer, the alveolar
damage caused by smoking, and its relationship with the onset of lung cancer.
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
7platform, integrated with AR and VR glasses and the medical Inter
net of Things, the virtual and the real cloud experts and terminal
doctors were able to communicate and interact in the metaverse
for medical education, science popularization, consultation, graded
diagnosis and treatment, and clinical research. Along with its
development, the application of the Metaverse in Medicine could
expand into comprehensive healthcare, not only enabling the vir
tual and the real cloud experts and terminal users (including ter
minal doctors, patients, and even their family members) to
interact, but also facilitating different medical services, such as dis
ease prevention, healthcare, physical examination, diagnosis and
treatment of diseases, rehabilitation, management of chronic dis
eases, in-home care, fifirst aid, and metaverse-assisted outpatient
attendance and consultation, etc.
Major clinical and non-clinical application scenarios of the
Metaverse in Medicine include: (1) research, (2) development of
computer software, (3) consulting, (4) science popularization, (5)
education and training, (6) clinical research (RCT, RWS, etc.), (7)
healthcare, (8) physical examination, (9) self-care and geriatric
nursing, (10) diagnosis and treatment of diseases, (11) drug and
device therapy, (12) surgical treatment, (13) hospital management,
(14) pharmacy, (15) quality control in medicine, (16) disease pre
vention, (17) insurance, (18) meeting, etc. Although trials have
only been carried out in a few scenarios at present, we believe that
it is just a matter of time before the metaverse is perfectly applied
in all these scenarios, with the solid technical foundation of the
MIoT and the metaverse. If we move with the times and work
against the clock, we will be able to accelerate progress towards
achieving our targets.
Leveraging the high technologies of the Metaverse in Medicine
in these application scenarios will also contribute to fulfifilling our
vision of benefifiting the general public. Moreover, it is noteworthy
that security is a prerequisite of the Metaverse in Medicine, and a
reliable security system is the foundation to ensure the normal
operation of such a platform. Availability, confifidentiality, integrity,
and controllability should be fully considered in the design of a
comprehensive security system to ensure physical security, system
security, operational security, and management security.
Author contributions
All the authors make a substantial contribution to this manu
script. DY, ZJ, CP, NC and CB participated in drafting the manu
script. DY, JZ and CB wrote the main manuscript. All the authors
discussed the results and implication on the manuscript at all
stages.
Declaration of Competing Interest
The authors declare that they have no known competing fifinan
cial interests or personal relationships that could have appeared
to inflfluence the work reported in this paper.
Acknowledgements
Not applicable.
Availability of data and material
All relevant data and material are presented in the main paper.
Funding
Science and Technology Commission of Shanghai Municipality
(20DZ2254400, 21DZ2200600, 20DZ2261200), National Scientifific
Foundation of China (82170110), Shanghai Pujiang Program
(20PJ1402400).
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
References
- The Metaverse Has Already Arrived. Here’s What That Actually Mean. Time.
Retrieved 2022-1-16.
- Wu J, Cao Z, Chen P, He CC, Ke D. User information behaviour from the
metaverse perspective: framework and prospects. J Inf Resources Manag. 1-17.
- Liu ZH. The Metaverse: an advanced form of human digital existence. Fresh
Reading. 2021;9:78–79.
- Choi HS, Kim SHJIJoIM. A content service deployment plan for metaverse
museum exhibitions—Centering on the combination of beacons and HMDs.
2016; 37(1pt.B): 1519-27.
- Bourlakis M, Li PFJECR. Retail spatial evolution: paving the way from traditional
to metaverse retailing. 2009.
Liu Y. The metaverse among disruptive technologies: Is it really valuable or not.
China Business. 2021;10:30–31.
Brennen B, dela E, Cerna. Journalism in second life. Journalism Stud. 2010;11
(4):546–554.
Yu GM. The Evolution Logic of Future Media: The Iteration, Reorganization and
Sublimation of ‘‘Hu-man Connection” – From the ”Age of Context‘‘ to the
”Metaverse‘‘ to the Future of the ”Mental World”. Press Circles. 2021;10:54–60.
- Yang DW, Zhou J, Song YL, Bai CX. Metaverse in Medicine. Clinical eHealth;
epub ahead of print.
- M. O’Brian K. Chan ‘‘EXPLAINER: What is the metaverse and how will it work?”.
ABC News Associated Press. Archived from the original on 4 (28 October 2021).
- Shaw DK. Overview of telehealth and its application to cardiopulmonary
physical therapy. Cardiopulmonary Phys Ther J. June 2009;20(2):13–18. https://
doi.org/10.1097/01823246-200920020-00003. PMC 2845264. PMID 20467533.
- ‘‘Apple Health guide: The powerful fifitness app explained”. Wareable. July 11,
- Retrieved August 22, 2020.
- Yang DW, Zhang J, Bai CX. Progress and prospect of the medical internet of
things. Int J Respir. 2012;32(18):1438–1441.
- Dorje T, Zhao G, Scheer A, et al. SMARTphone and social media-based Cardiac
Rehabilitation and Secondary Prevention (SMART-CR/SP) for patients with
coronary heart disease in China: a randomised controlled trial protocol. BMJ
Open. 2018;8(6).
https://doi.org/10.1136/bmjopen-2018-021908. PMID:
29961032; PMCID: PMC6042601 e021908.
- Yang X, Xie RH, Chen S, et al. Using video feedback through smartphone instant
messaging in fundamental nursing skills teaching: observational study. JMIR
Mhealth Uhealth. 2019;7(9):15386. https://doi.org/10.2196/15386. PMID:
31489839; PMCID: PMC6786856.
- Yun K, Chu Z, Zhang J, et al. Mobile Phone Intervention Based on an HIV Risk
Prediction Tool for HIV Prevention Among Men Who Have Sex With Men in
China: Randomized Controlled Trial. JMIR Mhealth Uhealth. 2021;9(4). https://
doi.org/10.2196/19511. PMID: 33847597; PMCID: PMC8080142 e19511.
- Navarro MA, O’Brien EK, Hoffman L. Cigarette and smokeless tobacco company
smartphone applications. Tob Control.. 2019;28(4):462–465. https://doi.org/
10.1136/tobaccocontrol-2018-054480. Epub 2018 Jul 20 PMID: 30030406.
- Chang S, Pierson E, Koh PW, et al. Mobility network models of COVID-19
explain inequities and inform reopening. Nature. 2021;589(7840):82–87.
https://doi.org/10.1038/s41586-020-2923-3. Epub 2020 Nov 10 PMID:
- Park S, Kim SP, Whang M. Individual’s social perception of virtual avatars
embodied with their habitual facial expressions and facial appearance. Sensors
(Basel). 2021 Sep 6;21(17):5986. https://doi.org/10.3390/s21175986. PMID:
34502877; PMCID: PMC8434682.
- Ashton K. That ‘Internet of Things’ Thing. RFID J. 2009.
- Who’s Who. ATS NEWS | VOL.35 NO.7/8.
Li SQ, Bai CX. Expert consensus on applying the internet of things for the
diagnosis and treatment of sleep-related breathing disorders. Chin J Asthma
(Electronic Version). 2013;02:5–8.
Bai CX. Practical Medical Internet of Things. Beijing: People’s Medical Publishing
House; 2014.
Bai CX. Guidelines on Applying Medical Internet of Things for the Graded Diagnosis
and Treatment. Beijing: People’s Medical Publishing House; 2015.
Bai CX, Zhao JL. Medical Internet of Things. Beijing: Science Press; 2016.
Themuller C, Health BC. 4.0: How Virtualization and Big Data are Revolutionizing
Healthcare. Springer International Publishing; 2017.
Sadoughi F, Behmanesh A, Sayfouri N. Internet of things in medicine: A
systematic mapping study. J Biomed Inform. 2020;103 103383.
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
- 8 Council OoSaTPNE. Strategy for American Innovation. 2014.
- Chan YY, Wang P, Rogers L, et al. The Asthma Mobile Health Study, a large-scale
clinical observational study using ResearchKit.
Nat Biotechnol.
2017;35:354–362.
- Intel. Center for Aging Services Technologies. Available from: https://
leadingage.org/center-aging-services-technologies.
Tachakra S, Wang XH, Istepanian RS, Song YH. Mobile e-health: the unwired
evolution of telemedicine. Telemed J E Health. 2003;9:247–257.
Hou W, Azizimanesh A, Sewaket A, et al. Strain-based room-temperature non
volatile MoTe(2) ferroelectric phase change transistor. Nat Nanotechnol.
2019;14:668–673.
Anliker U, Ward JA, Lukowicz P, et al. AMON: a wearable multiparameter
medical monitoring and alert system. IEEE Trans Inf Technol Biomed.
2004;8:415–427.
Siontis KC, Noseworthy PA, Attia ZI, Friedman PA. Artifificial intelligence
enhanced electrocardiography in cardiovascular disease management. Nat Rev
Cardiol. 2021;18:465–478.
- Castillo-Secilla J, Olivares J, Palomares J, Soto-Hidalgo J, Gámez J, Tapia L.
ZIGBEE PULSE OXIMETER. 2011.
- Toshiba. Toshiba’s AI offers advice on improving habits toward reducing risk of
lifestyle diseases. 2020. Available from: https://www.global.toshiba/ww/
news/corporate/2020/10/pr1501.html.
- Bai CX. IoT-aided three-plus-two model for the differential diagnosis of
pulmonary nodules. Int J Respir. 2014;1201–1202.
- Lung Cancer Study Group of the Chinese Thoracic Society, the Chinese Alliance
Against Lung Cancer Expert Group. Chinese Expert Consensus on the Diagnosis
and Treatment of Pulmonary Nodules (2018 edition). Chinese Journal of
Tuberculosis and Respiratory Diseases 2018;41:763-771.
- Bai CX. Early screening and management of lung cancer: a weapon to solve the
problem of lung cancer in China. Int J Respir. 2019;39:1601–1603.
- Chinese expert group on the IoT-aided assessment and management of
pulmonary nodules. Chinese Expert Consensus on Applying the Internet of
Things as Assistive Technology for the Assessment and Management of
Pulmonary Nodules, Int J Respir, 2022, 42(1): 5-12. DOI: 10.3760/cma.j.
cn131368-20211110-00835.
Tong L, Yang DW, Bai CX. Revelation of American lung cancer prevention and
control to China. Int J Respir. 2021;41(5):321–324.
Niederman MS, Richeldi L, Chotirmall SH, Bai C. Rising to the challenge of
COVID-19: advice for pulmonary and critical care and an agenda for research.
Am J Respir Crit Care Med. 2020;201:1019–1022.
Cai BQ, Cai SX, Chen RC, et al. Expert consensus on acute exacerbation of
chronic obstructive pulmonary disease in the People’s Republic of China. Int J
Chron Obstruct Pulmon Dis. 2014;9:381–395.
Bai C. Letter from China. Respirology. 2018;23:718–719.
Su X, Bai CX. Leveraging cloud computing and terminal to embrace the new era
of medical Internet of Things. China Med Pharm. 2016;6:1–3.
McGinnis JM, Fineberg HV, Dzau VJ. Advancing the learning health system. N
Engl J Med. 2021;385:1–5.
Grieves M. Virtually Intelligent Product Systems: Digital and Physical Twins, in
Complex Systems Engineering. Theory and Practice; 2019.
Sim I. Mobile Devices and Health. N Engl J Med. 2019;381:956–968.
Pennisi E. Pocket DNA sequencers make real-time diagnostics a reality. Science.
2016;351:800–801.
Yang DW, Bai CX. The current state and prospects of applying the Internet of
Things in medicine. China Medical News.. 2021;36(19):1.
Wang Y. How to develop holographic information. Secretary’s Companion.
1997;2:708–710.
Hao YT, Jin Y, Fan XM, Pan J, Yan J. Method of Virtual Product Development
Based on Holographic Product Model [J]. Comput Integr Manuf Syst.
2003;9:357–362.
- Wang BY, Sun QY, Ma DZ, Huang BN. A cyber physical model of the energy
internet based on multiple time scales. Automat Electric Power Systems 2016;40
(17):13–21.
- Freese P. Introducing Holographic Emulation. Unity Blog in Technology.
September 29, 2016.
Yang LY, Chen SY, Wang X, Zhang J, Wang CH. Digital twins and parallel
systems: State of the art, comparisons and prospect. Acta Autom Sin.
2019;45:31.
Chen C, Wang JX, Chen C, Sha K. Applications and prospects of holographic
projection in medical fifield and medical education. Chin J Med Educ Res.
2020;19:3.
Yao ZT, An W, Turdi M. A study of the application of virtual simulation based on
MOOCs model for undergraduate clinical teaching in maxillofacial surgery
Course. Educ Res. 2021;3.
Yang K, Li Z, Yuan YF, Chen ZQ, Lei H, Wang XH. Application of Stepped Training
for the Surgical Robots. China Higher Med Educ. 2018;3.
BingBing Zheng DY. 3D gray density coding feature for benign-malignant
pulmonary nodule classifification on chest CT %. J Med Phys. 2021;1–11.
- Ding Q, Liu JW, Hong NJ, Chen HW, Yu R, Li DM. Augmented Reality. Scientifific
and Technological Innovation (previously Heilongjiang Science and Technology
Information). 2015:1.
Huang J, Han DQ, Chen YN, Tian F, Wang HA, Dai GZ. A Survey on Human
Computer Interaction in Mixed Reality. J Computer-Aided Des Comput Graph.
2016;28:12.
Zhao ZD, Zhang GR. Augmented reality: progress and prospects. Computernik.
2018;23:243–245.
- Chinese expert consensus on diagnosis and treatment of pulmonary nodules
with medical Internet of Things. Int J Respir. 2017; 37:8.
- Ma HL, Zou SB. Human-computer integration: the future of artifificial
intelligence. Information China. 2017;3.
- Yang, J. Zhou, R. Chen et al.
Clinical eHealth 5 (2022) 1–9
9
