(Bloomberg) — International Business Machines Corp. will invest “hundreds of millions a year” into a free data analytics technology, according to Beth Smith, general manager of the company’s analytics division.
The move, announced by the company Monday, follows the hiring of about 20 people for a new technology center in San Francisco to improve and extend a University of California at Berkeley-developed free software project named Spark, which lets people analyze large amounts of data more rapidly than with other technologies such as Hadoop. Bloomberg reported the creation of the technology center last week.
Smith said that over time she expects the center to grow to 300 people, along with other teams in the company’s laboratories around the world.
As part of the effort, IBM will make a research project named SystemML free. This lets people spot patterns in data via a technology called machine learning, on top of Spark. It bears similarities to a project called MLlib, which is backed by DataBricks Inc., a Spark-focused company spun out of Berkeley by some of the academics behind the project.
“It will not be a competition,” Smith said. “The design point is one that does not cause overlapping features, functions, projects.”
