{"id":18287,"date":"2024-08-05T18:27:13","date_gmt":"2024-08-05T17:27:13","guid":{"rendered":"https:\/\/telecomkh.info\/?p=18287"},"modified":"2024-08-05T18:27:38","modified_gmt":"2024-08-05T17:27:38","slug":"together-with-ai-sweden-fraunhofer-iais-gains-large-scale-computing-capacities-for-training-large-language-models","status":"publish","type":"post","link":"https:\/\/telecomkh.info\/?p=18287","title":{"rendered":"Together with AI Sweden, Fraunhofer IAIS gains large-scale computing capacities for training large language models"},"content":{"rendered":"

The Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS and the NLU group of AI Sweden have jointly received computational capacity on the new high-performance computer MareNostrum 5 at the Barcelona Supercomputing Center<\/strong><\/p>\n

This is one of the largest contingents granted by the The European High Performance Computing Joint Undertaking (EuroHPC JU) for the development of European large language models (LLMs) on the EuroHPC infrastructure. The \u201cEuroLingua-GPT\u201d project will run for one year. This means that large European multilingual open source models are now in sight.
\nThe contingent approved via a EuroHPC \u201cExtreme Scale Access\u201d comprises 8.8 million GPU hours on H100 chips and has been available since May. \u201cThese computing capacities are a milestone for Germany and Europe. The models trained with it will massively accelerate the use of generative AI in companies and give both business and science a boost \u2013 GenAI ‘made in Europe’ is thus becoming a reality,\u201d says Dr. Joachim K\u00f6hler, head of the NetMedia department at Fraunhofer IAIS. With the new computing capacities, small models in the range of 7 to 34 billion parameters and large models with up to 180 billion parameters can be trained from scratch.<\/p>\n

One model family, all European languages \u2013 Fraunhofer IAIS and AI Sweden combine their expertise<\/strong>
\nThe new EuroLingua models are based on a training dataset consisting of 45 European languages, dialects and codes, including the 24 official European languages. This gives a significant weight to European languages and values \u2013 multilingual large language models are still rare. Training will start at the end of May 2024 and the first joint models are expected to be published in the coming months.
\nProject leader Dr. Nicolas Flores-Herr, team leader Conversational AI at Fraunhofer IAIS says: \u201cThe goal of our collaboration with AI Sweden is to train a family of large language models from scratch that will be published open source.\u201d Magnus Sahlgren, head of Research NLU at AI Sweden adds: \u201cBoth the public and private sectors in the EU are asking for open, powerful language models trained for European languages. This is one way to meet that need.\u201d
\nThe models developed on the EuroHPC infrastructure are intended on the one hand to serve as generalist basic models to support research and science, and on the other hand \u2013 for example in joint transfer projects \u2013 to provide specialized models for specific sectors or areas of application for productive use in companies or public administration.
\nTo achieve this, the two organizations are pooling their expertise: Fraunhofer IAIS andAI Sweden’s NLU group are two of the leading LLM labs in Europe with proven expertise and years of experience in developing LLMs. For example, Fraunhofer is leading the OpenGPT-X consortium project funded by the Federal Ministry for Economic Affairs and Climate Protection (BMWK), in which large European, multilingual open source models are also being developed. The NLU group at AI Sweden has developed the GPT-SW3 LLM for the Scandinavian languages. The two teams are also working together on other open source community projects. EuroLingua-GPT is also one of three major ongoing EU projects on language models in which Fraunhofer IAIS and AI Sweden collaborate.<\/p>\n

Above, 8.8 million hours of computational capacity for Fraunhofer IAIS and AI Sweden on the new high-performance computer MareNostrum 5 f at the Barcelona Supercomputing Center: This is one of the largest contingents granted for the development of European large-scale AI language models (LLMs) on the EuroHPC infrastructure. \/ \u00a9 Photo: By courtesy of Barcelona Supercomputing Center<\/em><\/span><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

The Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS and the NLU group of AI Sweden have jointly received computational capacity on the new high-performance computer MareNostrum 5 at the Barcelona Supercomputing Center This is one of the largest contingents granted by the The European High Performance Computing Joint Undertaking (EuroHPC JU) for the … <\/p>\n

Continue reading \u00abTogether with AI Sweden, Fraunhofer IAIS gains large-scale computing capacities for training large language models\u00bb<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":18288,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/posts\/18287"}],"collection":[{"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/telecomkh.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=18287"}],"version-history":[{"count":1,"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/posts\/18287\/revisions"}],"predecessor-version":[{"id":18289,"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/posts\/18287\/revisions\/18289"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/telecomkh.info\/index.php?rest_route=\/wp\/v2\/media\/18288"}],"wp:attachment":[{"href":"https:\/\/telecomkh.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=18287"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/telecomkh.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=18287"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/telecomkh.info\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=18287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}