Not logged in.
Quick Search - Contribution
Contribution Details
| Type | Master's Thesis |
| Scope | Discipline-based scholarship |
| Title | Quantifying the Trustworthiness Level of Federated Learning Models |
| Organization Unit | |
| Authors |
|
| Supervisors |
|
| Language |
|
| Institution | University of Zurich |
| Faculty | Faculty of Business, Economics and Informatics |
| Date | 2022 |
| Abstract Text | In the last decade, the rise of Deep Learning (DL) in the development of Artificial Intelligence (AI) has greatly improved the performance of AI models which are becoming increasingly relevant as a support to the human decision-making process. With the ever widening spread of AI applications powered on Big Data, centralized machine learning became challenging due to the existing data silos in many industries where data contain sensitive information. The rising concern for data privacy in AI is promoting the development of privacy-preserving Machine and Deep Learning (ML/DL) techniques such as Federated Learning (FL) where model training is performed collaboratively by distributed data contributors in a decentralized manner. FL enables data privacy by design since local data are not exposed. The increasing interest and adoption of FL systems prompt the need to investigate the ability to trust the decisions made by FL models as compared to centralized machine learning. There is a large body of existing literature on the topic of Trustworthy AI where the requirements are drawn out for an AI system under the five pillars of trust: i) robustness, ii) privacy, iii) fairness, iv) explainability and v) accountability. These pillars were developed in the context of traditional ML/DL systems. As the attention of AI shifts to FL, more efforts are needed to identify trustworthiness pillars and evaluation metrics relevant for FL models. This work analyzed the existing requirements for trustworthiness evaluation in AI and adapted the pillars and metrics for state-of-the-art FL models. A comprehensive taxonomy for Trustworthy FL is proposed as a result of the analysis. Based on the taxonomy, an evaluation algorithm, FederatedTrust, was designed and implemented as a third-party Python library which can be imported as a plugin to an FL development framework to evaluate the trustworthiness level of FL models. The FederatedTrust library harnesses the meta data and configuration settings of FL models gathered from the development framework and generates inputs and outputs for trustworthiness analysis based on the metrics identified in the taxonomy. At the end of an FL training, a report containing the trust scores of each metric and pillar that make up the aggregated trustworthiness level is generated for the FL model created. The report helps to identify the areas impacting trust within the model configuration and execution so that improvements can be made to make the model more trustworthy. Validation of the algorithm was conducted in the form of experiments to test the usefulness of the trustworthiness report generated by FederatedTrust under different FL settings. Observations and discussions were made on the experiment results to analyze what can be improved in the future development of this evaluation framework for Trustworthy FL. |
| PDF File |
Download
|
| Export |
BibTeX
|