Video: How do you add efficiency in AI models? First, look where people are looking.
What you get, starting out in this video, is that algorithms impact our lives in, as CSAIL grad student Sandeep Silwal puts it, “silent ways”
Silwal uses a simple example – maps – in discussing what he calls the “marriage of provable algorithm design and machine learning.”
Lots of people, he notes, want to move from the area around MIT, south across the Charles to Fenway Park, to see the Red Sox.
That sort of fact could inform the thinking about how to program algorithms. For example, Silwal mentions how you can analyze data results to identify the most visited websites on the Internet – and direct focus accordingly.
“We use (algorithms) to compute fundamental things about us,” he says. “And the nice thing about algorithms is that we can optimize them for … various resources that we care about.”
What do people want? Well, as Silwal points out, they want fast-loading maps. (We literally want maps to load in milliseconds when we’re trying to use GPS and navigation, hopefully not barreling down the road at 60 miles an hour, but more safely, pulled over to the shoulder.)
We also, he adds, want space for storage.
Silwal’s world is math, and for algorithms, math correctness, he says, equals the right answer.
You want your mail sorted correctly – that’s the example he gives, explaining that consistency is a must-have, that you need to get the right answer not once, but every time you generate a user-driven event.
“These algorithms are run every day by millions of people all around the world,” he says. “And we don’t know what they’re going to ask for. So we want it to be correct.”
Now here, Silwal also talks about ‘worst-case guarantees’ or fundamental reliability for algorithms. (Papers like this one give more of a detailed explanation of how to use worst-case guarantees in practice.)
In his presentation, he talks about working on a new paradigm called learning augmented algorithms:
“So the idea was, can we diverge this information that we know from the past (or from any other data that we have,) to actually change the algorithm design process itself? So we want this feedback loop between data and algorithm design … And what that means is: can we look at the inputs or the instances that we’ve run the algorithm (on) in the past, and extract patterns and other information out of it, to help ‘seed’ our algorithm or ‘warm start’ our algorithm, or even change the algorithm design itself, so that it’s better tailored to the patterns that we observed in the inputs that we actually run it on?”
Take, he says, the exodus from MIT every weekend of people, again, going south:
“You can say, ‘Okay, I know that 90% of people always want to go south,’” he explains. “So first, I’ll just load the part of the map that corresponds to that direction, and I’ll search that direction first. And if it’s actually true that 90% of people want to go south, then we’ll end up finding our shortest route much faster than if we’ve loaded the whole map beforehand. And this saves valuable time and computational resources.”
Using that idea – everybody wants to go to Fenway Park, but nobody wants to go to (x), we can uncover a lot of efficiencies. That’s it in a nutshell, although math does factor in.
Silwal points out two desirable outcomes in this line of work.
One is accuracy.
The other is, again, worst-case guarantees: making sure that you will get the result you need from the algorithm, even in more of a “black swan” query.
“Looking more towards the future,” Silwal says, “this is a very new area, at least in the algorithm world. And it’s new, because we want to take a more principled approach of having both provable guarantees, while also taking advantage of all the great and exciting progress that we’re making in the machine learning world.”
Later, in closing, Silwal talks about how this can help with climate change, in terms of saving computational resources. And he reminds us to stretch as we sit through a dozen or more mini-lectures a day! But this was one you didn’t want to miss, if you’re in the world of, as he mentions, algorithms and machine learning. Some of the other talks brought ideas relevant to this area of AI, too. We’re taking notes and bringing all of this to a wider audience, where the sort of work mentioned here is instrumental in building “AI 2.0”.
Read the full article here
