Synergos Is Officially Launched!

In previous posts (here and here), we gave a preview of AI Singapore’s Federated Learning platform Synergos. Today we are happy to announce that Synergos is officially launched!

We begin with a recap of what Synergos is and its main components.

The motivation for Synergos

Data is at the core of machine learning. Nevertheless, in many real-world projects, a single party’s data is often insufficient and needs to be augmented with data from other parties. However, there are also many concerns (regulatory, ethical, commercial etc.) stopping parties from exchanging data.

An example can be found in the healthcare domain. Individual hospitals alone typically have a limited amount of local data to build a robust model. There are existing studies (this one as an example) showing the benefit of using data from more hospitals to build models. Nevertheless, even though hospitals are convinced of the value of sharing data, there are a lot of regulatory concerns stopping hospitals from sharing data, since healthcare data is usually viewed as sensitive personal data in either general or sectoral data protection regulations, e.g. GDPR or HIPPA.

Federated Learning is an emerging privacy-preserving machine learning technology. It enables multiple parties holding local data to collaboratively train machine learning models without exchanging their data with one another, hence preserving the confidentiality of different parties’ local data. 

The design of Synergos

Synergos is a platform which AI Singapore has been building to make Federated Learning more accessible and sustainable. The diagram below gives an overview of the key components of Synergos. For a detailed description of each component, please refer to this post. 

Broadly, these components are grouped into three layers of functionalities.

1. Federated training, whose aim is to make Federated Learning simple and user-friendly.
2. Model management, whose goal is to make Federated MLOps simple.
3. Platform management, whose goal is to make Federated Learning sustainable.

Synergos makes Federated Learning accessible

In conventional machine learning, it is commonly assumed that all data are independently and identically distributed (IID). Or in simple words, it assumes that all data are from the same generative process and the generative process does not have memory of past generated data. However, in Federated Learning, as different parties do not really see other parties’ data, it cannot be assumed that they all follow the same generative process. Special care is needed to address such non-IID data. Otherwise, the model derived with Federated Learning may not converge and generalise to different parties’ data or take longer to converge. Many federated aggregation algorithms have been proposed to address this problem. 

Synergos makes Federated Learning user-friendly and accessible, taking away the burden from the users in implementing those federated aggregation algorithms. In Synergos, the Federation component implements a number of those algorithms. The most basic aggregation algorithm is FedAvg. Besides this, the current version of Synergos also supports more advanced aggregation algorithms, including FedProx, FedGKT, etc. More aggregation algorithms will be supported in future versions.

Synergos further reduces the burden with the Orchestration component, which supports auto-tuning of multiple Federated Learning models with different configurations of aggregation algorithms, aggregation settings, model hyper-parameters, etc.

Synergos makes Federated Learning sustainable

Usually, different parties incur non-negligible costs in acquiring and cleaning their data. They rarely altruistically share their data with others and risk losing their competitive edge. These parties would be more motivated to share their data when given enough incentives, such as a guaranteed benefit from the collaboration and a fair higher reward from contributing more valuable data. Otherwise, without any party motivated to contribute data, it could be detrimental to the sustainability of Federated Learning. 

Synergos makes Federated Learning sustainable by building the Contribution & Reward component to evaluate contributions and reward different parties fairly based on their contributions.  

We are still actively working on this component, and it is not yet available in the current version. We plan to implement model reward, which is the outcome of a research supported by AI Singapore. Conventionally, reward is associated with monetary gain. While this remains a natural and viable option, there are scenarios where monetary returns are not preferred or even impossible. Innovatively, Model reward rewards the participating parties with models of different quality based on their contribution, instead of the usual monetary rewards.

The path forward

The launch today is not the end of the story. Rather, it is the start of a long journey. Moving forward, besides the Contribution & Reward component, there are already a few enhancements and new features planned, including:

• Support of non-neural network models. 

Currently, those federated aggregation algorithms implemented in the Federation and Federated Grid components mostly support deep neural networks. Nevertheless, there are still many commonly used machine learning models that are not neural networks based. 

In the next version, we plan to integrate outstanding research outcomes in the field of Federated Learning to support more aggregation algorithms, e.g. Federated GBDT (SimFL), etc.

• Support of Vertical Federated Learning

The first post of this series discussed two common paradigms of Federated Learning: Horizontal Federated Learning and Vertical Federated Learning. 

Horizontal Federated Learning is useful in scenarios where different parties have a big overlap in the feature space (columns) but small overlap in the user space (rows).

Vertical Federated Learning is useful in the scenarios where different parties have a big overlap in the user space (rows), but a small overlap in the feature space (columns).

In the current version, Synergos only supports Horizontal Federated Learning. We are also working on the support of Vertical Federated Learning. 

• Integration with other compute and/or storage engines. 

Compute & Storage is an interface to compute and storage backends, which different parties use in local training. The current version of Synergos supports data that is managed by a local file system and S3-compatible storage, and the compute load is handled by a single node. 

We are actively working on support for other storage services and compute frameworks in the future versions, e.g., Spark, Horovod. 

• Support of privacy-enhancing technologies (PET) 

In Federated Learning, what is exchanged among parties is mainly the intermediary model learnings like gradients and/or weights. This protects different parties’ local data since it does not require sharing of raw data. Nevertheless, exchanging of gradients could also lead to information leakage. 

In future versions, we will support application of privacy-enhancing technologies (PET) like homomorphic encryption (HE) or secure multi-party computation (SMPC) to better protect participating parties’ data.

In summary, some of the planned enhancements and new features are as follows:

“Synergos” is a Greek word. The English word “synergy” was derived from “synergos”, which means “to work together” or “to cooperate”. We therefore also invite you to work together with us in this journey. 

Check out our code repositories at GitHub, start using it as a tool in your machine learning toolbox, and contribute your code to the platform. Synergos adopts a modular design. Different components are maintained in separated code repositories. Check out the key components like Synergos TTP and Synergos Worker. As a quick start, Synergos Simulator allows you to run all different configurations of Synergos in a sandboxed environment from your local computer. User guide is available here. 

Do share with us your feedback and suggestions on new features or areas we could improve. Please also join the discussion in our community group.

The Federated Learning Series

• AI Singapore’s Journey Into the World of Federated Learning
• Voices from the AISG Federated Learning Lab
• A Peek into Synergos – AI Singapore’s Federated Learning System
• Do Great Things Together With Federated Learning
• Synergos Is Officially Launched! (this article)