Follow-up interview after the Wise.io acquisition: Industrial AI at GE Digital, autoencoders for building training datasets, and combining physics-based models with data-driven ML.
“One of the distinctions I make between the consumer Internet of Things and the industrial Internet of Things is that when your Fitbit breaks you call up customer support and you sort of complain, but when your jet engine has a problem that can have some serious consequences. And so the stakes are a lot higher.” – Joshua Bloom
“You're in this very interesting dance where it's lots of data, yet in some sense it's a small data problem because you only get the sort of bad or rare anomaly events every now and then. And even when you get those, they're all sort of Anna Karenina, like they're all unhappy families, they're all different in their own way.” – Joshua Bloom
“I think of it as physics models only getting us 90% of the way there to a great answer, and then adding a sort of data-driven layer on top of that is the path that we're seeking.” – Joshua Bloom
“Your life as a person depends upon the industrial Internet of Things, and it's a great time to be part of that, and there's a massive amount of work to be done.” – Joshua Bloom
[Music] Hello and welcome to another episode of TWiML Talk, the podcast where I interview interesting people doing interesting things in machine learning and artificial intelligence. I'm your host Sam Charrington. If you get my newsletter you already know this, but last week we hit a major milestone for the podcast. I'm excited to share that thanks to you, we've served up over 500,000 plays of this show. We're super grateful to everyone who's ever listening to the show, send us feedback, or engage with us via the site or social media. Thanks also to all of our guests, especially those who started out as listeners and later became guests like Evan Wright and Sarah Guo. We've come a long way in a short amount of time and we couldn't have done it without you.
Next up, we're ready to announce the first winner of our AI conference giveaway. Drum roll please. Congratulations to Shinu from New Haven, Connecticut. Shinu was one of only 5 people to complete every possible method of entry and clearly it paid off. Shinu, we look forward to seeing you in San Francisco. We're still working to finalize our second winner, so stay tuned to our Twitter feed at @twimlai for updates. If you didn't win the contest but still want to join us at the artificial intelligence conference in San Francisco, head over to twimlai.com/goaisf and enter code PCTWIML for 20% off the cost of most packages.
Next, we're less than a week away from the first ever meeting of our online paper reading group, the TWiML online Meetup. Our first discussion will be on the recent paper from Apple, “Learning from Simulated and Unsupervised Images through Adversarial Training.” If you haven't registered yet, head over to twimlai.com/meetup and you can do so there. The meetup will be Wednesday August 16th at 11 a.m. Pacific time. It will be recorded for those who can't participate live. If you've already registered but haven't read the paper yet, now's the time to get started, and if you have started reading and have questions, please post them over on the Meetup Slack channel which you should have been invited to after registering.
Before we get to the main event, I'd like to give a quick shout out to our friends over at Bonsai for their continued support of the podcast and our industrial AI series. Bonsai offers an AI platform that lets enterprises build and deploy intelligent systems for industrial applications. If you haven't investigated the company and their platform before, I think you'll find it interesting. You can find more information about them and their early access program at bonsai.ai/twiml.
Finally, about today's show. I recently had a chance to catch up with a friend and friend of the show, Josh Bloom, vice president of data and analytics at GE Digital. If you've been listening for a while you know that Josh was on the show around this time last year just prior to the acquisition of his company Wise.io by GE Digital. It was great to catch up with Josh on his journey within GE and the work his team is doing around industrial AI now that they're part of one of the world's biggest industrial companies. We talked about some really interesting things in the show, including how his team is using autoencoders to create training data sets and how they incorporate knowledge of physics and physical systems within their machine learning models. Of course, Wise.io at GE Digital is a sponsor of my industrial AI research and this podcast series, and for that we're extremely grateful. To learn more about Wise, visit their site at wise.io. To learn more about Wise.io at GE Digital visit their site at wise.io.
All right everyone, I am on the line with Joshua Bloom. Josh is VP of data and analytics at GE Digital, and if Josh's name sounds familiar it's because it should. He was our fifth guest here on the show and that was back in September of 2016. Josh, welcome back to the show, you're our first repeat guest.
Awesome, thanks for having me.
I'm super excited to have you on the show, and I encourage folks who haven't already listened to that, or even haven't listened to it recently, to go back and listen to it again. It was TWiML Talk 5 and it's been one of our most popular shows of all time, so you can really dig deep into Josh's background by going back and listening through that show. But for now, why don't you give us a little bit of your background and catch us up on what you've been up to recently.
Yeah, so I started off in physics and astrophysics and did a PhD at Caltech, went to Harvard as a postdoc, and then got really interested in robotization. It was then that I started getting excited about machine learning as I realized that we had some big data problems coming down the pike when we needed to do discovery on new images, let's say coming from telescopes. At that point my colleagues were basically saying I would just hire more grad students to look at the data. So that's how I came upon machine learning a little over 10 years ago. Fast-forward a bit, we wound up building up an end-to-end system to do some astronomy projects with data and machine learning, and then with the research group wound up starting a company called Wise.io based out of Berkeley, California. Over time we wound up building out a set of products in customer support, integrating with Zendesk and Salesforce to help support agents basically become more efficient and delight their customers even more. Last year, around the time that we had our first conversation, Sam, we had some broad interest from a number of different companies, and GE Digital wound up becoming very interesting and compelling for us, and I'm sure we'll get into that. But we announced our acquisition by GE Digital in November of last year in 2016, and since then have been working within GE Digital for a much broader GE ecosystem and GE's customers, and certainly happy to talk to you about what we've been doing and how we've made the transition from really customer facing and consumer internet to industrial internet.
Great, that's exactly what I want to focus on for this show. And in fact we can start out by me thanking you and GE for graciously sponsoring our industrial AI, both the research and the podcast.
Well it's our pleasure, and needless to say it's been fun for me personally and for those around me at Wise within GE Digital to also have you go through the same sort of process as we
wind up learning about this industrial machine learning world. In some sense we've lived sort of parallel tracks, coming to realize how important it is, what kind of value there is, and how different it is from the other types of machine learning and AI that's done still in industry but not in the sort of hardcore machine industrial context.
So maybe let's start there with the kinds of things you've been learning. I can't think of a more target-rich environment. The way I think you've once explained your role at GE is to kind of AI all the things, or at least be a part of that, and there are certainly a lot of industrial things there to be AI'd, if you will — we just verbs AI, I think you get the credit for coining that, if that's right. How's that been going, and what have you been learning?
Yeah, in some sense the practical nature of what it means to do machine learning in production at scale with fault tolerance is the same sort of thing that we took from our work in previous product sets and applied it to the sorts of problems that GE has in front of it. And I'd say the most practical thing is to say that we shouldn't be AI'ing all the things, that a lot of things don't need AI. Oftentimes when I give talks internally within GE, one of the big things that I'll challenge people with is this notion that everything needs machine learning, or if all we did is just apply TensorFlow to it then all our problems would be solved. We don't believe that. I think that people who have been working in this field for a long time understand that very deeply. And so part of our mission is to help people within GE and eventually GE's customers to understand what are the workflows, what's the type of data where advanced analytics or machine learning or even more broadly AI can be used to affect better business outcomes. It's with that lens of why are you doing this that we're able to say yes to a bunch of things but say no to a bunch of other types of projects where traditional business rules may just make great sense, or these are problems that are massively complicated, these are machines that have very good physical models that describe future behavior based on past behavior and that's good enough given the business outcome.
But I'd say one of the things that's changed a lot for us is to understand how important this is, not just at the scale, not just in the details of what it means to get a right or wrong answer. For your listeners to understand, you're always trying to optimize something when it comes to accuracy, an area under a curve or some false positive rate at a fixed false negative rate. But as we start imagining — and we'll get I think into some of the details in this interview about some of the specific projects — you can imagine that there are new places on the ROC curve where you don't ever want to be wrong, where you always need to be right. And this changes the nature of how you do data science, it's changing the nature of how you build products around it. But I'd say the thing that we've learned is that that view of not everything needs machine learning to get a great business outcome, but everything needs to work within the current context of not just an old industrial company like GE which has been around for 125 years, but with the business processes that have been built up in some of the oldest, most analog parts of the economy. Let alone not being digital, let alone not being software savvy, trying to bring products into a place that is sort of used to doing things with the status quo because they just work is a challenge in and of itself. And all of that in some sense makes great sense when you understand that what we're talking about is machines that affect our lives. One of the distinctions I make between the consumer Internet of Things and the industrial Internet of Things is that when your Fitbit breaks you call up customer support and you sort of complain, but when your jet engine has a problem that can have some serious consequences. And so the stakes are a lot higher, and because of that there's a whole regulatory environment which is something that very few people in the consumer internet have had to deal with. Yes, you sort of have to think about PII, you have to think about HIPAA compliance and things like that, but now if you're talking about a regulatory oversight body with the FAA or the FDA, there are some extra boundary conditions that are put upon us as we start thinking about bringing machine learning into those sorts of worlds.
Your team isn't really targeting or chartered with kind of making sweeping revolutionary changes at GE, but rather you're proudly taking a much more incremental approach.
You have to start somewhere, and again taking a very practical view of what it means to transform an existing workflow that involves data, that involves people, that involves physics based models, that involves decision rules, and then start bringing that into a machine learning centric workflow. There are a lot of different stakeholders involved, and many times people have already tried bringing machine learning in and failed for various different reasons. And so one of the sharp elbows that we wound up building up as an independent company, and we bring to GE now, is around that notion of what are the problems that you should be solving, and in particular should you be going after the highest value, most complex ones, or should you really just start somewhere. We really think about low-hanging fruit, and in some sense that's our lens: within the industrial context, where is the low-hanging fruit where it's so obvious that AI can have a measurable impact, not just on things like accuracy or time to make a prediction or something like that, but in real dollar terms. The way I like to talk about it is we're not trying to solve the problems that end with a B, they shouldn't end with a K. And so we're sort of in that M world where at the millions of dollars a year level, if we have impact there and we can start working with the individual business units and their customers to start helping people understand even how to structure a new problem around AI, and understand what it means to do data governance right, how you wind up basically building up an AI-first product from scratch, then we wound up winning because we get to multiplex across multiple internal and external customers.
One of the things that I specifically remember about this conversation we had on this point was you talked about kind of impact of 1% in your world.
Yeah, that's right. That really in some sense gets to the scale. If you have a 1% improvement in a product and a workflow that is making hundreds of K a year in revenue, that's not a really big deal, but if you do have a 1% improvement, let's say in efficiency, in a billion dollar product, that starts to get to be real dollars. So in some sense, what I just said before of taking the low-hanging fruit and not trying to solve the really big problems, we get away with within the context of GE just because the scale at which we're talking is just so immense. To just give you a sense
of it, for instance, when a jet engine finishes a flight, call it a five-hour flight on average, there's about a terabyte of data that's generated just from that one engine. And you can imagine even the process of offloading that data from the airplane after it's landed and then getting it into a data lake, that itself can be pretty complicated. But then doing sort of real-time analytics on that and making some decisions from a preventative maintenance perspective is one of the really important things that we have to be able to do. But now if you think about, well, there's 50,000 jet engines that are flying every day, that gives you just some sense of the enormity of the scale. So each flight is basically a day of Twitter data, and then you go a factor of a few orders of magnitude larger than that. So for us, yes, we get to work on the quote unquote low-hanging fruit with fairly large dollar numbers, in part because making small incremental improvements in the workflows that involve lots of data for very expensive, important machines is just sort of the reality of where the industrial internet is right now.
So can we talk a little bit about some of the use cases that you've seen? Are there ones that you can walk us through?
Yeah, so I can't go into all the specific details, but the one that I just spoke about within aviation is an important one, and something that's been discussed publicly is the need to have advanced analytics applied to data that's coming off of airplane engines to achieve better outcomes. One of the things that is important to recognize about many of these industrial use cases is there's a huge value to being able to understand ahead of time when something is going to break or whether something is in trouble. And that is where you get into some very interesting data science problems of, especially given a lot of these objects very rarely fail, how you build up sort of counterfactual evidence so that you can test your models offline. The easiest thing to do would be to build the machine learning model that says take every engine off the wing after every flight, and by golly you'd find every single problem because somebody would take it off, but then that whole industry would come to a halt. We would probably destroy the world's economy just given the extra latency of what it would take to retire an airplane every single day. So that obviously doesn't make sense. So there our false negatives would be basically zero but our false positives would be just uncomfortably high. The other approach is to say everything's working all the time, and for the most part you'd be right. And the number that I have in my head is that the sort of failure modes are only, a few in a million flights will there be a significant problem with an engine, which is why we have multiple engines on a plane. And so you're in very, very small number statistics land, and you can't really ever know, if I said take this engine off the wing, whether it would have failed had I not said that. There may be some diagnostic evidence you could see when you actually look at it, but you can't ever gather the counterfactual of what if I didn't do this. And I can't really run an A/B test either, where you say, well, I think you should take these off the wing but I'm not going to say anything about that. That obviously has its own problems as well. So you're in this very interesting dance where it's lots of data, yet in some sense it's a small data problem because you only get the sort of bad or rare anomaly events every now and then. And even when you get those, they're all sort of Anna Karenina, like they're all unhappy families, they're all different in their own way. That's a real challenge from the data science perspective, and that's where some of the interesting innovation has to happen from an R&D perspective, is to work in this kind of really long tail world. So that's kind of one family of use cases that we're interested in, but I imagine before we get into other use cases, I'm sure people are asking, okay, how do you address that technically from a data science perspective, what are some of the ways that you tackle that problem?
Well, in some sense it comes back down to, do we even tackle that specific one, or do we tackle ones that are adjacent to that? And getting back to some of the work that we did in customer support — and I think this is one of the really core design principles of how you think about building machine learning products — that is building assistive tools that wind up not sort of making a decision by themselves but actually provide
information and insights to analysts who are looking at the data. And there are analysts who are looking at the results of data coming off of all these engines. So instead of saying yes, this is going to fail and take it off, or no it's not, there's an adjacent problem where you can wind up saying I'll create a ranked, prioritized list of the engines that I think an analyst might want to look at, and then you let the domain experts in that world go through and make some decisions on that. So it becomes kind of an accelerant and an efficiency play, rather than a black or white, almost draconian type of thing. So we're not anywhere near the point where machines are going to wind up generating work orders without any people in the loop.
So when you wind up pulling it back a little bit from “this is going to fail” and “this is okay” to “I think this is something somebody might want to look at” — it turns out that analysts look at lots of things, and so they're often digging into a specific engine to understand what's happening with it. And so we have a lot of that data from the past. Now it's not a very long tail problem, we make it sort of more evenly balanced of X percent look like they're fine just from the very high-level overview, and 1 minus X percent look like they need to get some more work done or somebody needs to dig into the data a little bit more. And as long as X is close to 0.5, or even if it's 0.1, you're in pretty good shape, because then you can apply sort of classic supervised machine learning to that. So that's sort of one trick that we have, is to take something out of the very sort of rare regime and try to bring it back into a world where it still has value, you could still measure that value, but it becomes kind of an empowerment tool rather than something that's making absolute decisions. That's kind of one thing that we've brought from our past lives into this one.
I guess the other one is the nature of the data is very different. You've got data coming from lots of different subsystems, it tends to be time series data. In the past we had worked a lot with natural language processing, and one thing that's very powerful with this sort of data is that if you have a lot of it, there are techniques one can use, let's say within the deep learning world, where you can in a very unsupervised way build up some capability of generating features out of that and then do anomaly detection off of those features or just do direct classification off of those features. And so you get to leverage lots of the quote unquote normal data and normal behavior and then use that to be able to make inference on the things that look like they're out of band. So for us, being able to apply some of the cutting-edge techniques in, let's say, recurrent neural nets and unsupervised learning around these time series data sets is very interesting.
And I guess the third part of that, which is coupled to the other two, is trying to do this all at scale and trying to do this all as real-time as needed for the specific problem. And we've really crossed over in terms of our back end in terms of what's needed from sort of large single machines in a multi-core environment to a multi-node type of environment. So doing deep learning across multiple GPU instances is something that our infrastructure that we had built before has had to adapt to.
What I heard was you talked about using deep learning to generate features that would allow you to train more traditional supervised models, and that reminds me — you may not be aware of this, but we're starting a paper reading group associated with the podcast, and the first version of this meetup is going to be on the 16th of August, and the paper that we're going to go through is one of the papers from CVPR where a team at Apple basically used the generative adversarial network to generate data to then train a supervised model, I think supervised. Actually, I'm not clear on this because I haven't read the paper yet, but I will by the sixteenth. But the point that I wanted to pick out is I've heard this notion a couple of times around using deep neural networks to generate training data for either supervised or more traditional models a few times, and I wanted to make sure that that's what I heard you say you were doing, and also kind of get some feedback from you on how broadly applicable are you seeing that across the various use cases you're looking at. Are you doing a lot of that?
Okay, so to be clear, we're not using GANs to generate training data. There's another approach called autoencoders that allow you to generate features in an unsupervised sense. So it's adjacent, but it's quite a different problem. What I will say though in general about GANs — and there's only kind of a few papers out on this in this context — is that it's actually kind of interesting if you think about the data privacy and the sensitivity around some of the data that we have access to at GE. Passing data around, and clearly it needs to be done in a highly covered and highly regulatory-approved way, isn't always the best thing, and especially when
it's very large. So you can imagine that there are use cases where different groups within the same company may need to get access to data, but instead of sending the data itself over, you can imagine building a GAN that's able to generate data that's like the original data. So if you have very sensitive data that you don't want moving outside of your walled garden or your data lake, you can imagine building a GAN that essentially simulates that, and instead of handing somebody the keys of the original data you could hand them the keys to the GAN, which, if for some reason it fell into hands that weren't supposed to see it, could provably not be able to reproduce the original data. Yet folks who got access to this GAN would in principle be able to build machine learning models against that. And so I think it's a very interesting and clever way to start thinking about passing information about a set of data around without actually having to pass that data around. And so being able to build models that learn from different groups and their data without those groups having to share data amongst each other, or without having to aggregate it all into one physical place, is of great interest to us. And it's not just sort of an interesting thing to do, in many cases it's a necessity. If we want to build great models and we can't even see the data or we can't federate it into a single data lake, we have to have really clear paths to being able to do that. And again there's academic literature on this, but there's not a whole lot of work that's being done on this in practice. So that's kind of one interesting regime.
So you bring this up as I think an important one, and the other one that we touched on before at the top was around sort of the marriage of physics-based and data-driven models.
Yeah. Unlike again in the consumer internet, where you build a data-driven model around customer behavior or around actions or on sentiment, etc., you can try to build some sort of latent understanding about how the brain works, but there are very complex biological systems effectively that are giving rise to the data that you wind up trying to apply on. There is no physics behind recommendation engines, there's no sort of core principles there. Yet in the industrial world you've got jet engines, you've got MRI machines, you've got wind turbines, nuclear power plants, and these are all built up by physical objects that, if you knew all the physics of them, then you wouldn't need any data because you'd be able to predict exactly what's going to happen in the future. So again, for a preventative maintenance perspective, you'd be just fine. But as we all know, even in very simple physical systems we often don't know all the physics. And while GE has I think some of the world's experts in all the various different subdomains, in material science, etc., building up complex physics models, I think of it as physics models only getting us 90% of the way there to a great answer, and then adding a sort of data-driven layer on top of that is the path that we're seeking. So rather than what we did in the past, where you just take effectively a fully data-driven model to get your outcomes, we're quite interested in understanding how in a rigorous way do we combine the outputs of physical models essentially as
the inputs to data-driven ones, in addition to all the sensor data that you're getting. So I'd say that's a really critical distinction. It's also a huge amount of white space that we see in the industrial machine learning world.
And that's something that GE's been pursuing or evangelizing for a while through this notion of digital twin. Can you talk a little bit about that and the role that it plays in the work you're doing around ML and AI?
Yes. A digital twin, for those that haven't heard the term, is an idea, and an implementation of an idea, that every physical object should have a virtual version of it that could live in the cloud, or if it's very sensitive can live in an on-premise environment. And that digital version should be kept up to date with the physical version of it, and it should know about its maintenance history, it should know, in the context of an asset model, if this is a part in a large machine it should know about the machine itself. So it's a very base layer. I think of a digital twin as a digital representation of a physical asset and all the data that's available about it, both historically and then in real time. Where AI and advanced analytics comes in on top of that is to say, well, given all of this data, can I make a predictive statement about what's going to happen to the physical object by interrogating the digital version of that? So rather than having to ping a hard drive which is on the device itself and try to pull out data, we need strategies that take data from those edge devices, bring them into the cloud, and then it allows me in a more relaxed cloud environment to be able to ask questions of that, maybe take some actions based on it. And then the next step after that, of course, is to take the results of some of those predictions and push them back into the physical device itself and potentially even update things like configuration variables based on predicted outcomes. I don't think we're really there yet across a wide swath of GE assets, but my sense is that's where a huge amount of value winds up coming in, if you're able to build machine learning models now not just against this one twin but against all the twins in the same asset class, and use those models not just on one customer's data but across all customers’ data to be able to get better outcomes.
And so this is somewhat related to the role of simulation in building ML and AI systems for industrial applications in general, right? We talked previously in a conversation about how you can't just take the engine off and put it through its paces to generate data sets. You
know, what have you learned about the process of using simulation as a way to create these models?
Yeah, so to be honest I haven't learned that much. Doesn't mean I can't anticipate on it, I just can't say that I've learned a tremendous amount, just that those aren't the sorts of problems that we've been directly exposing ourselves to. Clearly there's a kind of reinforcement learning play in that conversation, about being able to simulate the environment or the results of an action that you wind up taking, and being able to build models offline before you wind up deploying it into the field. We haven't ourselves been working on sort of that kind of robotics angle, but that's obviously really important. That said, simulations get back to that physics-based model that I was describing earlier. In some sense I think of physics-based models as essentially simulation. You've got a simulation of your physical object because you think you understand most of the physics based on the whole history of what's happened to that object. If you've done your job with simulation, you should have some uncertainty bands into what state is happening next. Then again, could you build a machine learning model that's effectively taking the results of that simulation and then using that as your more or less physics-plus-data-driven model? Yes. We've got I think a sort of simpler notion of how you do that, which is just taking the base predictions out from the physics-based model and using that as inputs, in addition to all the sensor data, to build a machine learning model off of that.
Can you elaborate on what you mean by that?
Yeah, let me take an example. Let's say that you've got a wind turbine and you've got a prediction of what the winds are going to be in an hour from now, and let's say that there are configuration variables that one might want to set effectively in real time, essentially with the optimization goal of maximizing the energy output. So we think about it as rotating around to try to get the optimal wind direction. So now, based on some data that's coming in and predictions about the future based on what we know about the physics of the object that's going to be spinning and how long it takes for us to spin, you can imagine, given a set of inputs like what the weather is going to be and what the weather was an hour ago and how fast the turbine is spinning now, you could run it through effectively a physics simulation that says, if I turn at this amount I'm expecting to get this energy yield out, if I turn it by this amount I'm expecting to get this energy yield out. What I would posit is that one can take the results of those predictions, think about it very simply as an efficiency curve as a function of the azimuthal angle of the rotation of the blade, there's going to be a place where that's optimized. And clearly that number is going to be wrong because there will be other physics of that object, let's say that object's got a little kink in it or it's at a slight tilt, or the models of the winds are always systematically off by five degrees in the orientation. What I would then do is say, well, in the past we've had all these turbines choosing a next best
action for itself and those haven't been necessarily optimal. How could we take all of the data that we have coming in and build a model off of that to try to get a more optimal answer? And of course what you would do, in some sense because you have multiple turbines in the field, is try different potential outputs based on what the model actually predicts, and then as you get the results back you say, well, that looks pretty good. That's how you're effectively building up a continually learning model, you could call it a reinforced model, that you could then deploy and get better and better over time. So you use again the predictions from the non-machine-learning part of your model and use that as an input to the machine learning model.
That's a pretty fascinating take on reinforcement learning, right? We think about reinforcement learning as you've got these physical systems perhaps, that maybe super simplistic like an Atari game, or maybe a simulation of a robotic system that has some degree of fidelity to real life, and you're using the simulation environment as a way to give you feedback on what happens in real life. And so the model is kind of acting in this simulation environment. What you're describing is kind of flipping that on its head and making real life, your wind turbine farm, your simulation environment. In a sense — I guess not in the sense of simulation but in the sense of the environment in which your models take control, deliberate actions to try to minimize the error, maximize efficiency.
Yeah, and to be clear, I don't think that's a unique view that I have. One of, I'd say, the highest value results I've seen come out of Google's DeepMind is an optimization on HVAC usage in large compute environments. You can imagine you've got 17 different levers to push up and down, and there is no a priori understanding other than the physical thermodynamic physics of how a room responds to HVAC, other than to say that you've got a whole bunch of data coming in like, well, what's the server load on every single object, where is it located in my data center, and all I can do is move these 17 levers up and down. That's something where your simulated environment is actually the real environment, and getting to turn those levers up and down is something that you wind up learning how to do over time because you've got a very clear optimization metric, which is how do I decrease my energy costs of pumping AC into this room while still maintaining a level of reliability on the machines to not go over their expected heat loads. So I think that's a very clear example in an industrial context in some sense where that sort of notion of reinforcement learning winds up playing out.
To be clear though, I would say that reinforcement learning, as you've described and as I was describing in the industrial context, is really kind of one end of the spectrum of what we'll call continual learning. And the sort of other end of the spectrum is you build a model on static data, you deploy it into production, and you grab feedback of whether you're right or wrong, and then a year later you build another model based on that feedback and you deploy it. That's sort of a very gradual, punctuated continual learning. And then step farther forward into sort of what we were doing in Wise when we were doing customer support, where you wind up having a model that's rebuilt every day because the world of customer support is changing fairly rapidly, and those are deployed and it's taking all the feedback of what you've just learned over the last day. And then you can imagine another one where it's sort of a cybersecurity environment where you want to have a model which is updating itself based on different threats that are coming in, that could be an update on a minute timescale. The kind of continual, true continuous, real-time learning where you have these online models that are just getting better and better over time and adapting to changing
environment is a very natural place to be. And so I see that all as part of a continuum. Now, it has vast implications about the engineering behind it and even the data science and the certain techniques you would use, but conceptually I think it's all sort of very similar.
I think the distinction that I thought I heard there — and we can go back to the DeepMind HVAC example in the context of this continuous learning spectrum that you outlined — is clearly they're modeling a physical system, they're deploying models out to a physical system, they are continually optimizing this model and getting to a model that can control their seventeen levers in a way that produces optimal, or at least way better to use a technical term, output over a given period of time. What I thought I heard you describing in the wind turbine example was, if we kind of map that to this continual learning spectrum and say that their feedback loop is operating at such a scale that it's near continual, what I thought I heard you describing was almost like accelerated continual learning, meaning we take this model and then we push it out to the physical devices, again in this case the wind turbines, and direct them to act in specific ways, not to pursue the plan that is outlined in this model but to deviate from that in a way that we think will accelerate learning and thus produce a better model more quickly. Did I make all that up, or were you saying something like that?
No, I think you've got it. And the missing piece of why I see that connected to the HVAC example is that in principle — and I don't know this to be true or not — you also have a thermodynamic model of what would happen if you threw lever four up higher and increased the HVAC in that region of the room. You in principle could model that, and you can imagine that instead of just using the data coming off of the individual computers as the input, instead you could also use that data plus a thermodynamic prediction about what would happen if you made a change. And I agree this could actually be an accelerant, because in a world again where you knew all the physics, you wouldn't need a domain-driven model, you would just, a data-driven model, you would just take all the heat loads and you'd crunch some big supercomputer, which itself could add to heat load in the room, but then you'd wind up being able to say very precisely, if I change all the levers in every single possible combination, what is the optimal output. But that's obviously a very hard, intractable problem given the complexity of even a data server room. So I connect those two examples in part because they are physical systems, and in both cases you have the potential, whether you use it or not, of using the true thermodynamics and physical modeling of what the expected output should be, and instead of having to explore that space in a purely data-driven way, you then have the ability to explore it in a sort of simulated way and at least do an exploration in the real world around where you think the good answer is going to wind up being.
Is there anything that you've learned or any direction that you can point us in terms of
the very practical, tactical approaches to integrating the physics into the modeling process and the models themselves?
Yeah, in a time-series sense, if you've got a physical system that is behaving effectively like a sine wave and there's a, call it a linear oscillator, that's involved in producing the data, you can imagine fitting with your physical model — so this is if you have parametrized the data that you have to your physical model — and getting out internal parameters that more or less give you a good measurement of the past, and you can then use to make a prediction about the future. So again, if it's a sinusoidal model, you'd fit the spring constant and the mass of the object just to make it really simple, and then, if it's a perfect model and a particularly easy problem, you'd get basically just a set of residuals from your fit that's consistent with the noise properties of the data. But oftentimes in more complex systems you might have some residuals that are correlated in time and not zero and not consistent with the errors, which means that there are more things going on. So instead of building a machine learning model on this large sinusoidal wave, why not just build a machine learning model on the residuals of the data? And there you could then bring in other data points, you can bring in metadata. It becomes very powerful in some sense, we move away the signal that you know about and only model the signal that is unmodeled. If I've only got a certain number of data points and I've only got a finite-size model, if I don't imbue any physical understanding of this into what I've got, I basically now have to fit a sinusoid using my machine learning. Well, they can do that of course, but then you're using your power up in something that's knowable by other means. But imagine you measured that, and you said, well, I know it's a mass and a spring and I get these measurements, but boy, I can't predict the next time step, why is that? So you do what I just said, you subtract off essentially your physical model, and then what you wind up realizing is the residuals are growing in time. It's because you forgot to include friction. Well, now your domain-driven model is going to basically learn what the friction constant is, so that it winds up getting a better prediction when you combine both of those two together, and it may have had a harder time finding that if you just said I don't know anything about the system, let's just use pure data to figure it out. So I think the whole point here is that these are physical systems that have the potential to be modeled, and yet our modeling capability on the physics side is imperfect because we don't know all the physics yet, but that's clearly in some sense prior information that we should be using, and then removing that out of the original signal and then only trying to predict what those residuals are so that we get a better answer.
When I've talked to — to try to be more precise here — some of the folks from the deep learning perspective, they kind of say, to probably poorly paraphrase them in a way that they'd disagree with, forget about all the physical model stuff,
what's cool about this deep learning stuff is that it'll figure everything out, so why worry about trying to incorporate these models, let's just throw tons and tons and tons of data at this thing and the network will figure it out. And that's always been counterintuitive to me, and so I just wanted to kind of poke you at this a little bit to make sure it's really clear, and that us as a community are really clear, on why at least in this domain the physical models are important and can be very powerful.
Well, let's put ourselves in the minds of the people that made those sorts of statements. There's great evidence that they're correct. You have a whole history, decades long, in computer vision where people are trying to come up with essentially physical models of what it is that a machine is seeing, and building a very deep understanding based on our understanding of the physics of vision into being able to make predictions, being able to do segmentation, being able to make classifications. And then deep learning matures, the large large data sets, benchmark data sets, wound up coming out, and all of a sudden all the old models and all the old ways just fall by the wayside. So there's an example in the computer vision world, there's even examples in natural languages —
Yeah, I was going to say, in the NLP world, a famous NLP person I think from the 70s, “every time I fire a linguist my model improves,” right? This crazy notion now that, why do I need to have a complex understanding of how language works when in the end all I really need to be able to do is just throw massive amounts of data at a network that's capable of learning it? So there's certainly examples where physical modeling, or theoretically hierarchical models of how language works, just basically were inferior once you had enough data and you had sophisticated enough networks. But the operative word or phrase that you said is “tons of data.”
That's problem-relative.
Exactly, that's problem-relative. Again, let's come back to the jet engine example. We've got tons of data, and more data in jet engine world in principle than in any data lake of any computer vision researcher. So then you would say, well, we've got more data so you just throw it at it. Except in this case, as I was saying before, we have so few examples of things going wrong, because these engines are so good and so robust, that you have to appeal to physics in some sense, you have to look at a physical understanding of these objects and the different failure modes. Because in computer simulation land, or in just physical simulation land more broadly, you can test a whole bunch of different things that never get tested or seen in the real world, and so you can build off a whole bunch of failure mode environments that, if you start seeing something like that happen in real data, that becomes a trigger point and you say, look, we've got a problem that's upcoming, let's take care of it. Whereas that purely data-driven model, my hunch is at best it could say this is something we haven't seen before, but it can't tell you what's going to happen in the end because you don't have a predictive model, it's literally never happened before in any of the data you've ever collected, yet it is something that you could fit a physical model to and show, well, given all the data of what's happened the last 10 days, this outcome is now expected. And again, you won't be exactly right on those, but I would argue that, and those are great examples, your physics-based models are going to wind up trumping purely data-driven models. And in the end I think it's going to become clear that it's going to be the combination of both of those notions that will make the most powerful, most robust outcomes.
Great, well this has been an awesome follow-up discussion, and I'm super excited to have you as our first repeat guest back on the podcast. Is there anything else that you'd like to
leave our listeners with?
Well, first of all, I would love to be the first three-peat guest as well, so we can look forward to that in the future, and maybe somebody has a model for whether that may happen. But what I will say is it is a very interesting time to be crossing over from the consumer internet, working on valued problems for people and their interactions, to what we're working on at GE Digital, high-value problems for people living their lives. Ready to get on airplanes, your house is powered by a power plant somewhere, etc., you go to a doctor and generally GE machines are the things taking pictures of you and your insides. So your life as a person depends upon the industrial Internet of Things, and it's a great time to be part of that, and there's a massive amount of work to be done.
So I mean, you're hiring?
Oh yes, you're hiring, please. I would love to have any of your listeners contact me directly, email's easy, it's just josh.bloom at ge.com, or you can tweet at me, I'm just @profjsb, and I'd love to hear from you. I think the important other thing is it's not just, are you a machine learning expert in this one little realm of time-series, multispectral time series for blah blah blah, it's we're really looking for people that just know how to scale computation and work with data under very restrictive environments around security and governance. So for me it's exciting not just thinking about it from the ML perspective, but from the engineering perspective.
Fantastic. Well, thank you so much, Josh, it's great to catch up.
Great to catch up with you as well, thanks so much for having me on, and love the series and love what you've been doing.
Thank you. [Music] All right everyone, that's our show for today. Thanks so much for listening and for your continued feedback and support. For the notes for this episode, to ask any questions, or to let us know how you like the show, leave a comment on the show notes page at twimlai.com/talk/42. Thanks again to our sponsors Bonsai and Wise.io at GE Digital. For more information about Bonsai visit bonsai.ai/twiml, and for more on Wise visit wise.io. Don't forget to register for our upcoming online meetup at twimlai.com/meetup, and my newsletter at twimlai.com/newsletter. Thanks again for listening and catch you next time.