rulin shao
232404870
On the Adversarial Robustness of Visual Transformers
Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision. This work provides the first and comprehensive study on the robustness of vision transformers (ViTs) against adversarial perturbations. Tested on various white box and transfer attack settings, we find that ViTs possess better adversarial robustness when compared with convolutional neural networks (CNNs) We summarize the following main observations contributing to the improved robustness of ViTs: 1) Features learned by ViTs contain less low level information and are more generalizable, which contributes to superior robustness against adversarial perturbations. 2) Introducing convolutional or tokens to token blocks for learning low level features in ViTs can improve classification accuracy but at the cost of adversarial robustness. 3) Increasing the proportion of transformers in the model structure (when the model consists of both transformer and CNN blocks) leads to better robustness. But for a pure transformer model, simply increasing the size or adding layers cannot guarantee a similar effect. 4) Pre training on larger datasets does not significantly improve adversarial robustness though it is critical for training ViTs. 5) Adversarial training is also applicable to ViT for training robust models. Furthermore, feature visualization and frequency analysis are conducted for explanation. The results show that ViTs are less sensitive to high frequency perturbations than CNNs and there is a high correlation between how well the model learns lowlevel features and its robustness against different frequencybased perturbations.
ArXiv
Rulin Shao Zhouxing Shi Jinfeng Yi Pin-Yu Chen Cho-Jui Hsieh
230799559
Robust Text CAPTCHAs Using Adversarial Examples
CAPTCHA (Completely Automated Public Truing test to tell Computers and Humans Apart) is a widely used technology to distinguish real users and automated users such as bots. However, the advance of AI technologies weakens many CAPTCHA tests and can induce security concerns. In this paper, we propose a user friendly textbased CAPTCHA generation method named Robust Text CAPTCHA (RTC) At the first stage, the foregrounds and backgrounds are constructed with randomly sampled font and background images, which are then synthesized into identifiable pseudo adversarial CAPTCHAs. At the second stage, we design and apply a highly transferable adversarial attack for text CAPTCHAs to better obstruct CAPTCHA solvers. Our experiments cover comprehensive models including shallow models such as KNN, SVM and random forest, various deep neural networks and OCR models. Experiments show that our CAPTCHAs have a failure rate lower than one millionth in general and high usability. They are also robust against various defensive techniques that attackers may employ, including adversarial training, data pre processing and manual tagging.
ArXiv
Rulin Shao Zhouxing Shi Jinfeng Yi Pin-Yu Chen Cho-Jui Hsieh
229352447
Stochastic Channel Based Federated Learning With Neural Network Pruning for Medical Data Privacy Preservation: Model Development and Experimental Validation.
BACKGROUND Artificial neural networks have achieved unprecedented success in the medical domain. This success depends on the availability of massive and representative datasets. However, data collection is often prevented by privacy concerns, and people want to take control over their sensitive information during both the training and using processes. OBJECTIVE To address security and privacy issues, we propose a privacy preserving method for the analysis of distributed medical data. The proposed method, termed stochastic channel based federated learning (SCBFL) enables participants to train a high performance model cooperatively and in a distributed manner without sharing their inputs. METHODS We designed, implemented, and evaluated a channel based update algorithm for a central server in a distributed system. The update algorithm will select the channels with regard to the most active features in a training loop, and then upload them as learned information from local datasets. A pruning process, which serves as a model accelerator, was further applied to the algorithm based on the validation set. RESULTS We constructed a distributed system consisting of 5 clients and 1 server. Our trials showed that the SCBFL method can achieve an area under the receiver operating characteristic curve (AUC ROC) of 0.9776 and an area under the precision recall curve (AUC PR) of 0.9695 with only 10% of channels shared with the server. Compared with the federated averaging algorithm, the proposed SCBFL method achieved a 0.05388 higher AUC ROC and 0.09695 higher AUC PR. In addition, our experiment showed that 57% of the time is saved by the pruning process with only a reduction of 0.0047 in AUC ROC performance and a reduction of 0.0068 in AUC PR performance. CONCLUSIONS In this experiment, our model demonstrated better performance and a higher saturating speed than the federated averaging method, which reveals all of the parameters of local models to the server. The saturation rate of performance could be promoted by introducing a pruning process and further improvement could be achieved by tuning the pruning rate.
JMIR formative research
Rulin Shao Hongyu He Ziwei Chen Hanyu Liu Dianbo Liu
203837090
Privacy Preserving Stochastic Channel Based Federated Learning with Neural Network Pruning
Artificial neural network has achieved unprecedented success in a wide variety of domains such as classifying, predicting and recognizing objects. This success depends on the availability of big data since the training process requires massive and representative data sets. However, data collection is often prevented by privacy concerns and people want to take control over their sensitive information during both training and using processes. To address this problem, we propose a privacy preserving method for the distributed system, Stochastic Channel Based Federated Learning (SCBF) which enables the participants to train a high performance model cooperatively without sharing their inputs. We design, implement and evaluate a channel based update algorithm for the central server in a distributed system, which selects the channels with regard to the most active features in a training loop and uploads them as learned information from local datasets. A pruning process is applied to the algorithm based on the validation set, which serves as a model accelerator. In the experiment, our model presents equal performances and higher saturating speed than the Federated Averaging method which reveals all the parameters of local models to the server when updating. We also demonstrate that the converging rates could be increased by introducing a pruning process.
ArXiv
Rulin Shao H S Liu Dianbo Liu
204852171
Stochastic Channel Based Federated Learning for Medical Data Privacy Preserving
Artificial neural network has achieved unprecedented success in the medical domain. This success depends on the availability of massive and representative datasets. However, data collection is often prevented by privacy concerns and people want to take control over their sensitive information during both training and using processes. To address this problem, we propose a privacy preserving method for the distributed system, Stochastic Channel Based Federated Learning (SCBF) which enables the participants to train a high performance model cooperatively without sharing their inputs. Specifically, we design, implement and evaluate a channel based update algorithm for the central server in a distributed system, which selects the channels with regard to the most active features in a training loop and uploads them as learned information from local datasets. A pruning process is applied to the algorithm based on the validation set, which serves as a model accelerator. In the experiment, our model presents better performances and higher saturating speed than the Federated Averaging method which reveals all the parameters of local models to the server when updating. We also demonstrate that the saturating rate of performance could be promoted by introducing a pruning process. And further improvement could be achieved by tuning the pruning rate. Our experiment shows that 57% of the time is saved by the pruning process with only a reduction of 0.0047 in AUCROC performance and a reduction of 0.0068 in AUCPR.
ArXiv
Rulin Shao Hongyu He H S Liu Dianbo Liu
226781751
Medical Data Privacy Preserving: Stochastic Channel Based Federated Learning with Neural Network Pruning (Preprint)
Rulin Shao Hongyu He Ziwei Chen Hui Liu Dianbo Liu
233348039
Visceral Obesity Related Indices in the Identification of Individuals with Metabolic Syndrome Among Different Ethnicities in Xinjiang, China
Background Few studies have focused on the predictive ability of visceral obesity related indices for metabolic syndrome (MetS) especially in different ethnic groups. This study aimed to evaluate the applicability of visceral obesity related indices for MetS screening among three major ethnic groups living in remote rural areas of Xinjiang. Methods Based on multistage stratified cluster random sampling method, 3,192 Uyghurs, 3,054 Kazakhs, and 3,658 Hans were recruited from Xinjiang, China. The Joint Interim Statement (JIS) criteria were used to define MetS. The receiver operating characteristic curve (ROC) area under the ROC curve (AUC) and predictive value of each visceral obesity related index were used to evaluate the predictive ability of MetS. Results After adjusting for potential confounding factors, the lipid accumulation product (LAP) Chinese visceral adiposity index (CVAI) waist to height ratio (WHtR) and atherogenic index of plasma (AIP) were significantly correlated with MetS for each ethnic group, and the odds ratios (ORs) for MetS increased across quartiles. LAP was best able to identify MetS status in Kazakhs (AUC=0.853) and Uyghurs (AUC=0.851) with optimal cut offs being 36.3 and 28.2, respectively. Both LAP (AUC=0.798) and CVAI (AUC=0.791) most accurately identified MetS status in Hans, with the optimal cut offs being 27.3 and 85.0, respectively. Moreover, the AUC of the combination of these visceral obesity related indices is higher for each ethnic group. However, compared with LAP, the improved value of combined screening was not significant. Conclusion LAP had the best discriminative capability for the screening of MetS among Kazakhs, Uyghurs, and Hans. The screening ability of CVAI for MetS was similar to that of LAP in Hans. Thus, LAP may be a complementary indicator for assessing MetS in various ethnic groups.
Diabetes, metabolic syndrome and obesity targets and therapy
Yusong Ding Yinbao Shao Heng Guo Mulatibieke Keerman Jia-ming Liu Huili Si Shuxia Guo Ru-lin Ma
212751867
Controlling the emission frequency of graphene nanoribbon emitters based on spatially excited topological boundary states.
Graphene nanoribbons (GNRs) with atomically precise heterojunction interfaces are exploited as nanoscale light emitting devices with modulable emission frequencies. By connecting GNRs with different widths and lengths, topological boundary states can be formed and manipulated. Using first principles based atomistic simulations, we studied the luminescence properties of a STM GNR junction and explored the applications of these topological states as nanoscale light sources. Taking advantage of the ultrahigh resolution of the STM tip, direct injection of high energy carriers at selected boundary states can be achieved. In this way, the emission color can be controlled by precisely changing the tip position. The GNR heterojunction can therefore represent a robust and controllable light emitting device that takes a step forward towards the fabrication of nanoscale graphene based optoelectronic devices.
Physical chemistry chemical physics PCCP
Xiaoyan Wu Rulin Wang Na Liu Hao Lin Zou Bin Shao Lei Shao Chiyung Yam
235355151
A seven gene prognostic signature predicts overall survival of patients with lung adenocarcinoma (LUAD)
Background Lung adenocarcinoma (LUAD) is one of the most common types in the world with a high mortality rate. Despite advances in treatment strategies, the overall survival (OS) remains short. Our study aims to establish a reliable prognostic signature closely related to the survival of LUAD patients that can better predict prognosis and possibly help with individual monitoring of LUAD patients. Methods Raw RNA sequencing data were obtained from Fudan University and used as a training group. Differentially expressed genes (DEGs) for the training group were screened. The univariate, least absolute shrinkage and selection operator (LASSO) and multivariate cox regression analysis were conducted to identify the candidate prognostic genes and construct the risk score model. Kaplan Meier analysis, time dependent receiver operating characteristic (ROC) curve were used to evaluate the prognostic power and performance of the signature. Moreover, The Cancer Genome Atlas (TCGA LUAD) dataset was further used to validate the predictive ability of prognostic signature. Results A prognostic signature consisting of seven prognostic related genes was constructed using the training group. The 7 gene prognostic signature significantly grouped patients in high and low risk groups in terms of overall survival in the training cohort [hazard ratio, HR 8.94, 95% confidence interval (95% CI) [2.041 39.2] P 0.0004) and in the validation cohort (HR 2.41, 95% CI [1.779 3.276] P 0.0001) Cox regression analysis (univariate and multivariate) demonstrated that the seven gene signature is an independent prognostic biomarker for predicting the survival of LUAD patients. ROC curves revealed that the 7 gene prognostic signature achieved a good performance in training and validation groups (AUC 0.91, AUC 0.7 respectively) in predicting OS for LUAD patients. Furthermore, the stratified analysis of the signature showed another classification to predict the prognosis. Conclusion Our study suggested a new and reliable prognostic signature that has a significant implication in predicting overall survival for LUAD patients and may help with early diagnosis and making effective clinical decisions regarding potential individual treatment.
Cancer cell international
Aisha Al-Dherasi Qi-Tian Huang Yuwei Liao Sultan Al-Mosaib Rulin Hua Yichen Wang Ying Yu Xuehong Zhang Chaohua Huang Haithm Mousa Dongcen Ge Sufiyan Sufiyan Wanting Bai Yanyan Shao Yulong Li Jingkai Zhang Leming Shi Dekang Lv Zhiguang Li Quentin Liu
8525496
An error codes detection based background calibration for split SAR ADC
An error codes detection based background calibration technique is proposed to compensate for the non linearity of Split SAR ADC. The nonlinearity suffered from the CDAC mismatch degrades SAR ADC performance in terms of DNL and INL. Calibration block detects both the missing and wide codes at ADC output, judges and then feeds back to adjust the compensation capacitor. A 9 bit SAR ADC with split CDAC has been implemented in standard 65 nm CMOS technology. Simulation results show that it achieves a peak SNDR of 54.7 dB and 0.843 mW from a 1.2 V power supply. +0.20/ 0.11 LSB DNL and +0.15/ 0.15 LSB INL are improved after calibration.
2015 IEEE International Conference on Electron Devices and Solid State Circuits (EDSSC)
Rulin Huang Leilai Shao Wuguang Wang Guoquan Sun Xiaolei Zhu