Gideon Mann: BloombergGPT: A Large Language Model for Finance

Описание: 9th Annual Bloomberg-Columbia Machine Learning in Finance Workshop (May 19, 2023)
Hosted by Bloomberg, The Center for Artificial Intelligence in Business Analytics & Financial Technology at The Fu Foundation School of Engineering & Applied Science (SEAS/IEOR), and The Data Science Institute at Columbia University

Gideon Mann (Bloomberg): BloombergGPT: A Large Language Model for Finance

Abstract:
The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in the literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general-purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology.

Bio:
Gideon Mann is the head of the ML Product and Research team in the Office of the CTO at Bloomberg LP. At Bloomberg, he guides corporate strategy for machine learning, natural language processing (NLP), information retrieval, and alternative data. His mandate includes building AI infrastructure (from GPUs to NLP libraries), incubating new technology (e.g., large language models), and new businesses (e.g., Bloomberg Second Measure).

He has over 30 publications and more than 20 patents in machine learning and NLP. He’s served as a founding member of the Data for Good Exchange (D4GX). Before joining Bloomberg in 2014, he worked at Google Research NY, where his team carried out basic research, as well as developing machine learning products such as Colaboratory. He holds a Ph.D. from The Johns Hopkins University.

#MachineLearning #MLinFinance #BloombergGPT