Introducing FlowerLLM
Flower, in partnership with the CaMLSys lab at the University of Cambridge, has trained for the first time a 1.3 billion parameter LLM using a novel formulation of federated learning methods. The resulting LLM and companion methodology, that we call FlowerLLM, beats the previous record set by Google DeepMind by more than a factor of three and paves the way towards an era of increasingly democratic participation within foundation models.