Financial data specialist Bloomberg employs hundreds of data scientists to keep users hooked on its ubiquitous Terminals - the keyboards and monitors that give financial staff access to reams of market information.
In the background the Bloomberg Terminal crunches through 80,000 news wires, 4,000 FX feeds and 370 exchanges, driving around 60 billion data points a day to keep terminal users up to date on the global financial markets. The task for the data scientists at Bloomberg is to make this ocean of information discoverable and relevant for all of its 325,000 subscribers.
Gideon Mann is head of data science at Bloomberg. His role is to manage all of the data science that occurs across the large and varied organisation.
"The bulk of data science that happens is building products," Mann told Computerworld UK. "So my role ends up being mostly managing strategic, technical initiatives in basically three areas: natural language processing, search and machine learning that are embedded into products which serve the terminal."
The overall aim for Bloomberg is to have "any data that is relevant to the financial industry on the terminal in a normalised way," Mann said.
The Bloomberg Terminal
While many will think of Bloomberg as a media organisation, subscriptions to the terminal and all of the associated data services - bundled together for around £20,000 a year - is used by thousands of bankers, traders, analysts and financial reporters, and is core to Bloomberg as a business.
In terms of where the data science comes in, Bloomberg started by experimenting with machine learning for sentiment analysis nearly a decade ago. Mann admits that it took some time to get the organisation to fully commit to machine learning - the computer science technique of teaching a machine to learn and adapt on the fly as it is fed large volumes of data - but the success of this project legitimised it for upper management.
"It took a number of years before the company realised that this particular competency takes a while," he said. "Engineers can do it but it is not simple. So then the company started to commit to hiring and investing in quantitative programmers." Now Bloomberg has between one and two hundred data science specialists within the organisation, according to Mann.
Once the organisation had proved the usefulness of the technique, and built the skills in-house, it started to apply the technique to the internal search on the terminal to improve data discovery through better ranking algorithms.
A more recent project used computer vision to pick out data from tables embedded deep within financial reports and filings, a task that was once performed manually by programmers.
Sign up for Computerworld eNewsletters.