About
https://scholar.google.com/citations?user=d9qRaYkAAAAJ&hl=zh-CN
Experience
No experience added yet.
Education
No education added yet.
Skills
Other Publications
Smart clothing system with multiple sensors based on digital twin technology
IEEE Internet of Things Journal, Dec 2022
Smart clothing is widely used for social safety, health monitoring, and sports monitoring. Current research focuses on the use of various materials or sensors to implement smart clothes with different functions, which implies that the functionality of smart clothing depends on the number of sensors used. For existing smart clothing systems, the greatest attention has been given to information processing algorithms and assembly of sensors, and the interaction between users and systems is ignored. To address this gap, this article considers a multifunctional smart clothing system constructed with several sensors. The smart clothing system proposed in this article mainly consists of a hardware module and a software module. Four types of sensors are incorporated into the hardware module to monitor the heart rate, blood oxygen saturation, body temperature, locating information, and activity states; the software module includes the 3-D model based on the user and the feedback system based on digital twin (DT) technology. The DT technology can map the fundamental states of users in terms of the monitoring indices from the hardware module, and give correspondent advice to users. This novel smart clothing system overcomes the lack of an interaction function in existing methods and introduces DT technology into smart wearable devices for the first time.
VTON-SCFA: A virtual try-on network based on the semantic constraints and flow alignment
IEEE Transactions on Multimedia, Feb 2022
An image-based virtual try-on system transfers an in-shop garment to the corresponding garment region of a reference person, which has huge application potential and commercial value in online clothing shopping. Existing methods have difficulty preserving garment texture and body details because of rough garment alignment and imperfect detail-retention strategies. To address this problem, we propose a virtual try-on network based on semantic constraints and flow alignment. The key idea of the framework is as follows: 1) a global-local semantic predictor (GLSP) is proposed to generate a reasonable target semantic map, which clearly guides the correct alignment of the in-shop garment with the body and the generation of try-on result; and 2) a novel appearance flow-based garment alignment network (AFGAN) is proposed to align the in-shop garment with the body, which is important to preserve maximum garment detail and ensure natural and realistic warping; and 3) we propose a synthesis strategy to integrate the aligned garment and the human body to preserve maximum body detail for generating a realistic result and preventing cross-occlusion and pixel confusion between different body parts. Experiments on the existing benchmark dataset demonstrate that the proposed method achieves the best performance on qualitative and quantitative experiments among the state-of-the-art virtual try-on techniques.
Intelligent wearable system with motion and emotion recognition based on digital twin technology
IEEE Internet of Things Journal, Apr 2024
Intelligent wearable systems have been widely used in health monitoring, motion tracking, and engineering safety. However, the single function of current wearable systems cannot satisfy the requirements of complex scenarios, and the wearable systems cannot establish a relationship with the virtual 3-D visualization platform. To address these issues, this article proposes a novel intelligent wearable system with motion and emotion recognition. Multiple sensors are integrated into the system to collect motion and emotion information. In order to achieve accurate classification and recognition of multiple sensor information, we propose a novel human action recognition network called the three-branch spatial–temporal feature extraction network (TB-SFENet), which can obtain more robust features and achieve an accuracy of 97.04% on the UCI-HAR data set and 92.68% on the UniMiB SHAR data set. To establish the relationship between the real entity and virtual space, we use digital twin (DT) technology to establish the 3-D display DT platform. The platform enables real-time information interaction, such as activity, emotion, location, and monitoring information. Additionally, we establish the TGAM electroencephalogram emotion classification (TEEC) data set, which contains 120 000 pieces of data, for the proposed system. Experimental results indicate that the proposed system realizes virtual reality (VR) information interaction between the personal digital human and actual person based on the intelligent wearable system, which has great potential for applications in intelligent healthcare, VR, and other fields.
Multimodal Wearable System With Dual-Frequency Enhancement Network for Risk Recognition
IEEE Internet of Things Journal, Apr 2024
Intelligent wearable systems have been widely used in health monitoring, motion tracking, and engineering safety. However, the single function of current wearable systems cannot satisfy the requirements of complex scenarios, and the wearable systems cannot establish a relationship with the virtual 3-D visualization platform. To address these issues, this article proposes a novel intelligent wearable system with motion and emotion recognition. Multiple sensors are integrated into the system to collect motion and emotion information. In order to achieve accurate classification and recognition of multiple sensor information, we propose a novel human action recognition network called the three-branch spatial–temporal feature extraction network (TB-SFENet), which can obtain more robust features and achieve an accuracy of 97.04% on the UCI-HAR data set and 92.68% on the UniMiB SHAR data set. To establish the relationship between the real entity and virtual space, we use digital twin (DT) technology to establish the 3-D display DT platform. The platform enables real-time information interaction, such as activity, emotion, location, and monitoring information. Additionally, we establish the TGAM electroencephalogram emotion classification (TEEC) data set, which contains 120 000 pieces of data, for the proposed system. Experimental results indicate that the proposed system realizes virtual reality (VR) information interaction between the personal digital human and actual person based on the intelligent wearable system, which has great potential for applications in intelligent healthcare, VR, and other fields.
VTON-MP: Multi-Pose Virtual Try-On via Appearance Flow and Feature Filtering
IEEE Transactions on Consumer Electronics, Aug 2023
Multi-pose virtual try-on has become a research focus for online clothes shopping due to the fixed-pose virtual try-on methods that cannot provide a different pose try-on effect. The challenge of multi-pose virtual try-on is that the detailed information of a generated image is difficult to obtain in the pose transformation and garment distortion. To solve the issue, we propose a multi-pose virtual try-on method via appearance flow and feature filtering (VTON-MP). First, a segmentation generation network of 2D keypoints about the target pose is used to predict the body semantic distribution of the target pose. Second, the desired garment is distorted to correspond to the body posture using the appearance flow figure alignment network (AFFAN). Third, latent useless feature weights are restrained using a filtering-enhancement block (FEB), and effective appearance feature weights are enhanced. Finally, the spatial relationship of body parts in the resulting image is further optimized using spatially-adaptive instance normalization (SAIN). Compared to state-of-the-art methods of subjective and objective experiments on the MPV dataset, the proposed VTON-MP achieves the best performance in terms of SSIM, PSNR and FID. The experimental results demonstrate that the proposed algorithm can better retain image details (head, hands, arms, and trousers).
Phase Contour Enhancement Network for Clothing Parsing
IEEE Transactions on Consumer Electronics, Mar 2024
Clothing parsing is a challenging task. In this task, there are two main inherent challenges, which are the fine-grained parsing of small clothing components and the missegmentation of similar clothing categories. To address these challenges, this paper proposes a phase contour enhancement network for clothing parsing. It achieves improved accuracy while maintaining a relatively small network complexity. Leveraging the distinctive features of phase maps, we introduce the phase contour enhancement attention (PCEA) module to augment the encoder’s object edge information. To further enrich the model’s feature extraction capacity, we present the dilated convolution pyramid (DCP) module, combining it with a lightweight decoder to achieve improved global context modeling. The synergistic integration of the PCEA and DCP modules enhances the network with remarkable capabilities in capturing intricate contours, effectively surmounting the challenges of fine-grained parsing for small clothing components and mitigating the missegmentation of similar clothing categories. Through extensive experiments on the CFP, Modanet, and DeepFashion2 datasets and further expansion experiments on the LIP dataset, our method demonstrates outstanding performance across multiple evaluation metrics, surpassing state-of-the-art methods for clothing parsing. In conclusion, our phase contour enhancement network exhibits remarkable performance in both clothing parsing tasks and extended human parsing tasks.
ArmBCIsys: Robot Arm BCI System With Time-Frequency Network for Multiobject Grasping
IEEE Transactions on Neural Networks and Learning Systems, Jun 2025
Brain–computer interface (BCI) offers a direct communication and control channel between the human brain and external devices, presenting new pathways for individuals with physical disabilities to operate robotic arms for complex tasks. However, achieving multiobject grasping tasks under low signal-to-noise ratio (SNR) consumer-grade EEG signals is a significant challenge due to the lack of robust decoding algorithms and precise visual tracking methods. This article proposes, ArmBCIsys, an integrated robotic arm system that combines a novel dual-branch frequency-enhanced network (DBFENet) to robustly decode EEG signals under noisy conditions with the high-precision vision-guided grasping module. The proposed DBFENet designs the scaling temporal convolution block (STCB) to extract multiscale spatiotemporal features from the time domain, while the designed DropScale projected Transformer (DSPT) utilizes discrete cosine transform (DCT) to capture key frequency-domain features, significantly improving decoding robustness. We fine-tune the masked-attention mask Transformer (Mask2Former) model on the Jacquard dataset and incorporate the multiframe centroid-intersection over union (IoU) tracking algorithm to build visual grasp segmenter (VisGraspSeg), enabling reliable segmentation and dynamic tracking for diverse daily objects. Experimental validations on both self-built code-modulated visual evoked potential (c-VEP) dataset (1344 samples) and two public c-VEP datasets demonstrate that DBFENet achieves the state-of-the-art recognition performance, and the system integrates the DBFENet and proposed vision-guided module and ensures stable multiobject selecting and automatic object grasping in dynamic environments, extending promising applications in healthcare robotics, assistive technology, and industrial automation. The self-built dataset has been made publicly accessible at https://github.com/wtu1020/ ArmBCIsys-Self-built-cVEP-Dataset
BiaCanDet: Bioelectrical impedance analysis for breast cancer detection with space-time attention neural network
Expert Systems with Applications, Apr 2025
Breast cancer is a life-threatening disease and early detection plays a crucial role in improving survival rates. However, current detection technologies face challenges such as invasiveness, radiation exposure, high costs, and limited accessibility. To address these issues, a novel breast cancer detection system called BiaCanDet based on bioelectrical impedance analysis (BIA) signal and space–time attention network is proposed for the first time. The BiaCanDet system utilizes non-invasive BIA to measure signal changes in electrical impedance and phase angle (PHA) based on the normal and tumour breast tissue. We collect the first BIABC dataset using BIA devices, which includes 2,444 samples from normal, benign, and malignant breast tissues. Professional physicians collect and annotate the samples through repeated tissue sampling with needle electrodes, combined with pathological analysis to ensure accuracy. This paper proposes a novel space–time attention network (STABFNet) for the complex bioelectrical impedance signal to obtain robust cancer signal feature, which designs the impedance phase enhancement attention (IPEA) model to improve recognition performance. The experimental results on the BIABC dataset confirm that the STABFNet network achieve the 93.37%, 90.64%, 89.37%, and 89.90% in terms of accuracy, precision, recall, and f1-score, respectively. Additionally, the system also demonstrates real-time detection capabilities in the lightweight mobile terminal. The proposed system and methodology introduce a pioneering approach for breast cancer detection that is low-cost and convenient, making it a promising solution for widespread adoption that has the potential to revolutionize early breast cancer detection worldwide.
SuperLightNet: Lightweight Parameter Aggregation Network for Multimodal Brain Tumor Segmentation
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Aug 2025
Multimodal 3D segmentation involves a significant number of 3D convolution operations, which requires substantial computational resources and high-performance computing devices in MRI multimodal brain tumor segmentation. The key challenge in multimodal 3D segmentation is how to minimize network computational load while maintaining high accuracy. To address the issue, a novel lightweight parameter aggregation network (SuperLightNet) is proposed to realize the efficient encoder and decoder for the high accurate and low computation. A random multiview drop encoder is designed to learn the spatial structure of multimodal images through a random multi-view approach for solving the high computational time complexity that has arisen in recent years with methods relying on transformers and Mamba. A learnable residual skip decoder is designed to incorporate learnable residual and group skip weights for addressing the reduced computational efficiency caused by the use of overly heavy convolution and deconvolution decoders. Experimental results demonstrate that the proposed method achieves a leading reduction in parameter count by 95.59%, the 96.78% improvement in computational efficiency, the 96.86% enhancement in memory access performance, and the average performance gain of 0.21% on the BraTS2019 and BraTS2021 datasets in comparison with the state-of-the-art methods. Code is available at https://github.com/WTU-MIS-Laboratory/SuperLightNet.
Latent Diffusion-Enhanced Virtual Try-On via Optimized Pseudo-Label Generation
Proceedings of the AAAI Conference on Artificial Intelligence, Apr 2025
Efficiently applying fully supervised learning to virtual try-on tasks is challenging due to the lack of paired ground truth in available training samples. Recent works have achieved virtual try-ons by employing self-supervised learning-based inpainting paradigms. However, this approach is heavily dependent on the constraints of inpainting masks. An incorrect mask can mislead the generated results, while overly large mask areas can lose essential original information, thereby hindering the synthesis of high-quality results. To address these problems, we propose a latent diffusion model-based virtual try-on network that achieves fully supervised learning using the concept of cycle consistency and knowledge distillation. Specifically, we divide our approach into pretext and downstream tasks. In the pretext task, we generate a pseudo-label (pseudo-person image) to form paired training samples, which enables the downstream task to achieve fully supervised learning. To prevent the unreliable pseudo-person image from introducing irresponsible prior knowledge, we propose a noise-covering strategy, which aims at fully optimizing the pseudo-label to eliminate the impact of the incorrect inpainting mask as much as possible. Additionally, we propose a skin refinement loss to further enhance the generation of details in the skin region. Extended experiments demonstrate that our proposed method is superior to state-of-the-art methods.
Collaborator Notes
No notes yet.
Sign in to write a note.