DARPA Goes 'Meta' with Machine Learning for Machine Learning
June 20, 2016 | DARPAEstimated reading time: 2 minutes
Popular search engines are great at finding answers for point-of-fact questions like the elevation of Mount Everest or current movies running at local theaters. They are not, however, very good at answering what-if or predictive questions—questions that depend on multiple variables, such as “What influences the stock market?” or “What are the major drivers of environmental stability?” In many cases that shortcoming is not for lack of relevant data. Rather, what’s missing are empirical models of complex processes that influence the behavior and impact of those data elements. In a world in which scientists, policymakers and others are awash in data, the inability to construct reliable models that can deliver insights from that raw information has become an acute limitation for planners.
To free researchers from the tedium and limits of having to design their own empirical models, DARPA today launched its Data-Driven Discovery of Models (D3M) program. The goal of D3M is to help overcome the data-science expertise gap by enabling non-experts to construct complex empirical models through automation of large parts of the model-creation process. If successful, researchers using D3M tools will effectively have access to an army of “virtual data scientists.”
“The construction of empirical models today is largely a manual process, requiring data experts to translate stochastic elements, such as weather and traffic, into models that engineers and scientists can then ask questions of,” said Wade Shen, program manager in DARPA’s Information Innovation Office. “We have an urgent need to develop machine-based modeling for users with no data-science background. We believe it’s possible to automate certain aspects of data science, and specifically to have machines learn from prior example how to construct new models.”
D3M is being initiated at a time when there is unprecedented availability of data via improved sensing and open sources, and vast opportunities to take advantage of those data streams to speed scientific discovery, deepen intelligence collection, and improve U.S. government logistics and workforce management. Unfortunately, the expertise required to build useful models is in short supply. Some experts project deficits of 140,000 to 190,000 data scientists worldwide in 2016 alone, and increasing shortfalls in coming years. Also, because the process to build empirical models is so manual, their relative sophistication and value is often limited.
A recent exercise conducted by researchers from New York University illustrated the problem. The goal was to model traffic flows as a function of time, weather and location for each block in downtown Manhattan, and then use that model to conduct “what-if” simulations of various ride-sharing scenarios and project the likely effects of those ride-sharing variants on congestion. The team managed to make the model, but it required about 30 person-months of NYU data scientists’ time and more than 60 person-months of preparatory effort to explore, clean and regularize several urban data sets, including statistics about local crime, schools, subway systems, parks, noise, taxis, and restaurants.
“Our ability to understand everything from traffic to the behavior of hostile forces is increasingly possible given the growth in data from sensors and open sources,” said Shen. “The hope is that D3M will handle the basics of model development so people can apply their human intelligence to look at data in new ways, and imagine solutions and possibilities that were not obvious or even conceivable before.”
Suggested Items
Argonne, Toyota Collaborate on Cutting-Edge Battery Recycling Process
05/01/2024 | BUSINESS WIREThe U.S. Department of Energy’s (DOE) Argonne National Laboratory has recently launched a collaboration with Toyota Motor North America that could reduce the nation’s reliance on foreign sources of battery materials.
Real Time with… IPC APEX EXPO 2024: Outlining Rehm's Innovations and Global Presence
05/01/2024 | Real Time with...IPC APEX EXPOMichael Hanke, the chief sales officer for Rehm Thermal Systems, shares insights on the North American market, new machinery, and software development. He also discusses Rehm's turnkey solutions, process flexibility, global presence, and the dedicated team of 700 employees working on global solutions.
The Knowledge Base: A CM’s Perspective on Box Build Practices
04/30/2024 | Mike Konrad -- Column: The Knowledge BaseIn the ever-evolving landscape of electronics manufacturing, the box-build process stands out as a critical phase that bridges the gap between individual component manufacturing and the delivery of a fully functional electronic system. This intricate procedure, encompassing the assembly of everything from PCBs to wire harnesses and mechanical enclosures, demands a high level of precision, efficiency, and innovation. As the electronics assembly industry expands and diversifies, understanding the best practices within box-build assembly has become paramount for manufacturers aiming to stay ahead in a competitive market.
TSMC Certifies Ansys Multiphysics Platforms, Enabling Next-Gen AI and HPC Chips
04/30/2024 | PRNewswireAnsys announced the certification of its power integrity platforms for TSMC's N2 technology full production release. Both Ansys RedHawk-SC and Ansys Totem are certified for power integrity signoff on the N2 process, delivering significant speed and power advantages for high performance computing, mobile chips, and 3D-IC designs.
Koh Young Showcases Award-winning Inspection Solutions at SMTconnect with SmartRep in Hall 4A.225
04/25/2024 | Koh Young TechnologyKoh Young Technology, the industry leader in True 3D measurement-based inspection solutions, will showcase an array of award-winning inspection and measurement solutions at SMTconnect alongside its sales partner, SmartRep, in booth 4A.225 at NürnbergMesse from June 11-13, 2023. The following offers a glimpse into what Koh Young will present at the tradeshow: