How to extract real-time, actionable insight from noisy, incomplete streaming sensor data (earthquakes, supernovae, traffic), and the interplay between domain scientists, statisticians, and computer scientists.
Inaugural lecture of the series, recorded by CITRIS and followed by a faculty panel discussion.
“What I know is that there's data science, and then there's data-driven science, and I know that data-driven science is a thing. I'm a data-driven scientist, and so was Galileo when he had this very nice set of time-series data of what are now called the Galilean moons.” – Joshua Bloom
“The problem is that every star is variable in its own way.” – Joshua Bloom
“The universe decides they're going to throw these photons at us, and it's silly for us to not put more detectors on them.” – Joshua Bloom
“We need statisticians and computer scientists like never before. But I think the novelty squared statement is also that I believe that you need us as well, because we're creating real-world problems — although we're astronomers, not that real world — we're creating real-world problems that are novel.” – Joshua Bloom
“Part of the reason for the creation of the boot camp was essentially to better put my money where my mouth is — or my time where my mouth is; that's the same thing, actually.” – Joshua Bloom
PÉREZ: Hello, everyone. I think we'll try to follow the time-honored Berkeley tradition, but no worse than Berkeley time. Thank you everyone for coming here today. We're going to start a new lecture series with the support from the Vice Chancellor for Research Office and Kaya's indomitable logistics energy. Josh Bloom from the Astronomy Department is going to be our first speaker today, but we hope to make this a regular lecture series where we talk and we bring people from multiple disciplines, hopefully to see each other, to discuss things with each other about the problems — theoretical, practical, computational, scientific, from any perspective — that the flood of quantitative data is presenting to all of our fields. And so we're going to try to keep the format conducive to this kind of discussion. So we'll have a short talk by Josh, then we will have a panel discussion with a few members that I'll introduce later, and then we'll open it also for discussion with the rest of you. So start thinking about multiple questions, because this isn't going to be a lecture with three minutes for discussion, but rather lots of time.
So Josh Bloom — it's an honor and a pleasure to introduce him. He's a good friend and a colleague. Josh is a professor in the Astronomy Department who has done some incredible work, and you'll see some of those highlights today. Josh got his master's degree at Cambridge, his PhD at Caltech, has a CV the size of a phone book. He won the 2010 Newton Lacy Pierce award from the American Astronomical Society. He was both a Hertz and a Sloan Research Fellow. He's concluding his sabbatical right now with being promoted to full professor on campus, and I'd like to congratulate him for that.
And just to give you an idea of the kinds of things that Josh pulls off: a couple of years ago we were getting together in my office on a Monday morning to start working on a grant, and he shows up at 10 a.m. completely bleary-eyed. And I asked him, what's going on, what happened? He said, oh, it was a rough week, and I had to write two Science papers this weekend. And I said, what do you mean? He said, yeah, we collected some data three weeks ago, we spotted a very rare astronomical event, and we wrote two Science papers about it over the weekend. And I thought, that's kind of odd. And lo and behold, about three, four weeks later, Science lands on my desk, and there's two back-to-back papers from Josh. By the end of the year he had assembled 52 papers in the previous calendar year, six or seven of which, I think, were Nature, Science, PNAS.
And yet he's deeply committed to the educational mission. We've been having a bunch of discussions lately about data science, and I'm always thrilled to hear how he keeps coming back to the question of how does the university do this kind of work while keeping the core of our educational mission up front. How do we engage new scientists? How do we engage the next generation of people for whom these tools are going to be critical? And together we're teaching a boot camp in the fall and trying to basically develop some of these tools for good research. So I'm going to leave you in his capable hands, and keep your questions in your mind, since there will be plenty of time for that. Thank you.
BLOOM: Thank you, Fernando. Thanks very much for the introduction. It's not often that I get to welcome people with my astronomy hat on, on a day of such great astronomical significance, and also of worldly significance: welcome to the first day of summer. It's the summer, in case you haven't been paying attention. I think this lecture series is incredibly timely. What I'm about to talk about is not really meant to be all that exhaustive in introducing you to the kinds of things that we think about in the context of the time domain in astronomy. It's actually really not meant to be all that didactic. I really want this to be considered as a warm-up for this upcoming discussion. This is in some sense about a domain science that happens here at Berkeley.
And one of the things that we've all been thinking about recently is whether this thing that we're calling data science — it's clearly a thing, right, in the same way that big data is — but is it intellectual in and of itself, or is it a collection of hodgepodge tools that enables data science to happen? And we hope over this lecture series — obviously we're not going to answer this in the next 45 minutes or so — that we get a chance to start thinking about that collectively and have you as a community contribute your ideas to this. What I know is that there's data science, and then there's data-driven science, and I know that data-driven science is a thing. I'm a data-driven scientist, and so was Galileo when he had this very nice set of time-series data of what are now called the Galilean moons. His discovery was obviously fairly —
By the way, since Fernando brought up this thing about Science, I'll bring in a little anecdote from when I was a graduate student. There was a new postdoc in our group from the University of Washington, and I was asking him, what does it mean to have a really high-impact paper? How do I get my stuff into Science or Nature? And he said, well, the way that you do it — and if you look at most Science or Nature papers, this is what it is — you're expecting to see no things and then you saw one, or you're expecting to see one thing and you saw nothing. And what you wind up seeing is that that's the vast majority of the amazing Science and Nature papers that I've been involved in. If that's the case, then there would have been four new things when you're expecting zero for Galileo, so that probably would have been pretty highly cited.
But I wanted to introduce this just as an early example of astronomers thinking about the time domain. There's obviously some grand challenges — not where we're pointing our new instruments, instead of pointing it at the horizon, and then saying, oh, let's see what happens when I point it up, which is what Galileo did, right? Telescopes were invented for military purposes. And he said, what happens when I just point at things that should be around and shouldn't have anything else around it? And he found something that was completely unexpected. There are grand challenges in the time domain, in the context of being a data-driven scientist, that are obviously important. For instance, finding an Earth-like planet in a one-year orbit around a Sun-like star is one of these grand challenges, and the race is really on.
And this is hard not because it's hard to articulate — as an astronomer or anyone else, everyone else could have come up with this question — but when we finally have the instrument that's going to be able to give us the data to answer this question, which star is the one that has an Earth analog around us, the data don't look so great. And they look wonderful if you zoom out, but if you look at what the units are on the y-axis over here, this is in parts per million. So this is changes of brightnesses of stars that are basically far away from us within our own galaxy, and we're seeing these things on the scale of what the Sun does with star spots, and even at smaller levels. And an Earth moving in front of that Sun for just a brief period of time, waiting another year and having it come in front, and then waiting another year and having it come again — you're looking for signals of this occultation that are at a similar level to just the ordinary ripples that you get from ordinary quiet stars.
So this is a really great statistical problem, even though we can ask this in a fairly simple way: actually getting at this answer, and finding a really good set of candidates, and then convincing your colleagues that this is actually something that's statistically very important. And what's interesting about this is this is one of those examples in astronomy where computer scientists and statisticians are having a very big impact doing signal processing on this amount of data. And by the way, this is only of order a month or so of data. Kepler has been collecting this data on one-minute time scales for the last three years, so there's a tremendous amount, basically pointing in the same part of the sky, something like 150,000 stars. It's a lot of data, but more than just the amount of it, it's really just the quality of the data and getting into really the needle-in-the-haystack work.
So who is going to make this discovery, I think, is an interesting question. Is it going to be somebody from a stats department? Is it going to be somebody from a computer science department? And computer science is important in this context because you're having to do, in frequency space, millions or billions of trials on a single object. And if you do all those trials, that's obviously computationally expensive, and now a number of different groups who are looking at the same data are starting to bring in GPUs to try to speed this whole process up. So will it be an astro person, a stats person, or will it be a group of astro, stats, and CS that actually make this grand discovery that convinces everyone they have it? And I think a more important question is, does this actually really matter what that group is? But what's very clear is around the world, stats and computer scientists and astronomers are coming together and trying to answer these grand challenges in fundamental ways that really push the envelope, not just of astronomy but of these other disciplines.
So there is a meeting that's ongoing right now at SAMSI, which is the Statistical and Applied Mathematical Sciences Institute in the Research Triangle, and they are asking this question right now about this data set. So maybe we'll hear something about it in about a week or so. This is not really that close to the work that I do in astronomy. I've been interested in finding black holes and looking at the ways in which black holes belch and burp and change their — the hard part here is not finding one needle in the haystack, but actually identifying in a statistically rigorous way where you're going to use your precious resources to do follow-up spectroscopy, as it's called, of all the different candidates.
So here's an example of how a black hole on the other side of the universe winds up belching and burping. It's what we call a light curve that's similar to the one you saw before, but now they are much, much larger. They can be of order 10 or even 100 percent of the actual measurement, which gets us into another interesting statistical regime. And you see that the data is not regularly sampled anymore on one-minute time scales. It's sparsely sampled; it's irregularly sampled. And you notice we've observed this in five different filters, essentially different colors, and you notice sometimes we have some colors and sometimes we don't have other colors. And what others realized is that the way in which quasars burble along can be described as a damped random walk in a statistical sense.
And so we asked the question: given a large catalog of light curves of everything on the sky, can you identify and separate out the stars, which change in different ways or don't change at all, and all the burbling black holes? And so this was bringing in some interesting Bayesian theory to come up and ask this question in a way — I'll try to phrase it as well as I can now — essentially, given a light curve of a source, is it behaving like a damped random walk? And then if you can couch that in the form of probabilities, then you can see how well you're doing on known data sets. And what we're able to do is, just in the time domain, in a single color, we're able to separate out known quasars, or burbling black holes, from stars. And we had a really nice separation where if we make a cut somewhere between the blue and the red curve, we'd get a very nice complete sample and almost a 100 percent pure sample.
So that was a very nice example of bringing in a whole body of stats knowledge that I actually didn't have myself. I was fortunate enough to work with somebody named Nat Butler, who was a very stats-minded astronomer, and he was able to bring some of these tools to bear. But even if I was able to articulate that question and say, can we use the time domain only to figure out what's a quasar and what's a star, I was not trained in a way that would have allowed me to answer that question in the way that we were able to as a team.
Some of the other things I think about: trying to classify stars and different types of variables across a very large and ugly taxonomic tree that's been set up over the last 150 years or so of variable stars. There's about a hundred different types of variables, and sometimes things look the way that we observe them just because of orientation effects and what sort of wavelengths we're looking at them in, but sometimes they appear to change and act strange because of real intrinsic physical effects. The taxonomic tree that describes how variable stars are connected to each other is this interesting hodgepodge of really physically motivated and really empirically motivated statements.
But anyway, what you see here on the right-hand side is an example of four different light curves of different sources. Again, some of these are actually pretty regularly sampled, but you notice the second one from the bottom, so-called Beta Lyrae, doesn't have very good sampling at all. And you also notice the very different time scales that we operate on. We've got things at the bottom changing on hour time scales, up to days, up to months, and up to year time scales. So astronomers are really working on all these different time scales. It'd be tempting to say that if I just had a really good template library of all these different variables, all I'd have to do is just do some matching.
The problem is that every star is variable in its own way. And even if it truly belongs to a certain class, when you get into the nitty-gritty — perhaps if you observe it with Kepler — you notice that they're all different, right? The Anna Karenina of how the stars work, right? You remember that all happy families are happy in the same way, and all unhappy families are all unhappy in their own way. So all stars might actually fit into one nice class, but when you really get in the nitty-gritty, every single thing is different. And how do we deal with that in a statistically rigorous way is a really interesting question that I've been asking.
The problem is we don't even have templates for all of these different classes, or at least well-observed templates. Most of the stuff we have is noisy, and it has in some cases spurious data — that is, if you want to think of it in a statistical way, we've incorrectly estimated what our uncertainties are on some of this data. This data doesn't just come from the universe. This is derived from an observation from a telescope, which has its own inherent flaws. The light is passing through the atmosphere; that light has passed through all the junk in the universe to get to us. And when we detect this on typically a charge-coupled device, a CCD, or some other digital detector, we have to extract the time series out of what essentially amounts to an image. Sometimes we do a bad job with that.
And perhaps most important, when we're really thinking about doing inference — and inference that has needs for follow-up to really extract the most science, that is, to take action on those — the telltale signatures of that event may not have even happened yet. And so that's very interesting, I think, if you start thinking about how to follow these things up in a resource-constrained environment. So what I like about this problem is that it's hard from an astronomy perspective, but it starts really poking at what people are capable of doing in the machine learning space, in the statistical space.
And one of the things that I'm proud of is that we were able to get a group on campus and start working on this question, not just as astronomers but as — and there's a PhD that's coming out of the group from James Long. This is one of the figures from his thesis, where he wound up showing that if I train a classifier on three different classes of sources on the left-hand side, where I actually know the answer, and then I apply that same classifier — this is basically making hard cuts in two different feature-space numbers — I wind up doing a very poor job. So even though I have a perfect classifier on one satellite, if I then try to apply what I've learned from one satellite to another, it becomes essentially a terrible classifier.
And that's because the way in which we observe the universe actually winds up affecting how we think about it. And that's true in a Heisenberg uncertainty way, but it's also true in a pretty obvious way. As you see here, these are the same classes of stars, and yet when we observe them differently — that is, we take data on different time scales with different telescopes — we get different results. So I'll skip through this.
The last thing I was just going to point out is sometimes we really want to take action on the inferences that we're meant to derive out of the data. We have on the left-hand side an example of two so-called discovery images that were done by some machine learning codes that my group built up, where we're trying to basically find that thing that's in the center that's new — those are, in both of these cases, new — and try to do this in an automated way, and then use all the previous history from before we observed that part of the sky to then come up with a very quick statement about what we think this is. And if it's a variable star or it's some other type of transient event in our own galaxy, different groups within these large collaborations may get excited, but for the collaboration that I've been working most closely with, we were trying to basically follow these things up as quickly as possible.
So here's the hard part: when you start taking people out of the real-time loop and trying to do all this inference, you have to be really careful not to blow all of your resources on things that just turn out to be mundane, right? You really are looking for that next high-impact paper. So find the supernova and do it quickly — don't wait for a year and then go back and look at your data archivally. Then do follow-up, but don't follow up everything; be judicious about when you follow up. Then do some inference given all the follow-up data that you've gotten, and then maybe you win a prize.
So on the left-hand side we've got questions that I'm interested in, in the context of time domain and astronomy. What is truly new in the sky? We call that discovery. We're doing distributed processing at NERSC for that, and using some machine learning techniques as well. What's the origin? Once I found something and said, aha, this is interesting — again, so we're trying to do machine learning at scale. It's only going to get worse as we get more data. And then, where are the quasars, where are the planets? And there we have to bring in some pretty hardcore Bayesian stats modeling and advanced signal processing to answer this stuff.
I don't work all that closely with others in other domains. We're starting to think about working together, and we recently were awarded a grant to start thinking about how those types of light curves then start applying to other domains. And so seismology and neuroscience are — at least when I look at those graphs, I say that that could be a variable star of some sort, so there must be something that we can do together. And it's pretty clear we're not going to be able to share much across the domain in terms of influencing each other's science, but the techniques that we need all seem to have some sort of — and so I think what we'll be hearing about over the next half an hour or so is in that commonality, and what we can all be doing together to improve the domain science.
We've been building up the Berkeley Center for Time Domain Informatics, and even though this was mostly an astronomy-only endeavor, we decided that we weren't going to have astronomy in the name, because we realized that over time we wanted to try to branch out to other disciplines. So for me, this is the next big step for us. So let me end this presentation with just a challenge, and I'll try to phrase it as what I'll call the novelty squared problem. If I'm trying to do something novel in astronomy — let's say I'm trying to find the first Earth-mass planet around a Sun-like star in a one-year orbit — no one would ever argue that that's not a novel endeavor to go after. But thankfully, that also seems to be at least a novel problem in the stats and computer science realm as well. And so those different groups all coming together and doing something novel is the novelty squared.
The problem is this: if novelty's rare, if you square this — and you require that you bring in, say, computer science and stats and applied math to this problem, and it's not actually all that novel to those folks, and yet it's novel to a domain — is that really the kinds of endeavors that we want to be thinking of as data scientists? Is that truly a data-driven science? And are we just thinking about math and stats and applied math as just these toolkits that just live out there, and our whole job is just to find the right one? I would suspect that some of the best things that are going to happen on this campus are not going to be somebody from the computer science department teams up with somebody in seismology and helps the seismologists figure out something they didn't know before. It's going to be because the seismologist found a type of data and a type of question that wasn't answered by existing theory within, say, statistics. So I'll leave it with that, and look forward to your questions during the panel.
PÉREZ: Oh, thanks a lot, Josh, for that talk. And so now I'm going to very briefly introduce our three panelists, and we'll try to spend a few minutes in the panel discussion, then we'll open it to the floor. Kaya asked me to remind you all that right after we wrap it up here, there's going to be tea and coffee and cookies outside, so that you can spend some time maybe talking amongst yourselves. We really are trying to build up a community here on campus around these— So we have today Philip Stark from the statistics department with us; Michael Silver from the Vision Science and Neuroscience— Vision Science Program at the Helen Wills Neuroscience Institute; and Richard Allen from geophysics.
And rather than speak anymore, I'd like to ask each of you maybe to spend a couple of minutes telling us a little bit about the kinds of problems that you work on and that interest you, and trying to see how some of the things that Josh said may tie into either ideas that you have or problems where you think you could contribute something. And I'd also like to ask you, if you see anyone in the audience that you've either partnered with or that you would like to work with at some point, and maybe consider either because you think you can help them or because you think they can help you out — and maybe try to talk to them afterwards, or over tea and cookies.
STARK: So do you want us to focus on things to do with time series and spatial series and so on? Is that the focus?
PÉREZ: Sure, sure. But not necessarily. I want to leave it up to you. I don't want to be driving the—
STARK: Should I start?
PÉREZ: Sure.
STARK: So I had mentally prepared to talk about time series and spatial series; I'll go that direction. The kinds of problems that I've worked on that involve these sorts of tools are primarily in physical sciences: in seismology, geomagnetism, helioseismology, cosmology — microwave cosmology — and then some things in Fourier transform infrared spectroscopy for identifying rock. So a lot of the problems — what they have in common is you have a signal that isn't stationary, but you're trying to recover some stationary property of the object that underlies it. So for example, you want to know something about the interior structure of the Earth, but what you're observing are these transient things that happen after an earthquake. And the data reduction that's involved is partly trying to separate the stimulus from the response — from the system that is then responding to the stimulus. And a lot of the difficulty in this is simplifying the physics to the point that you can actually do something, and then doing that separation of what for you that day is signal from what for you that day is noise.
In seismology, traditionally, the data reduction problem was really addressed in a very draconian manner. We just saw Josh put a seismogram up there, and the only feature that people cared about is where did the big wiggles start, right? So you reduce this enormous time series to one piece of information: where did the wiggle start? So one of the things I'm worried about is, to what extent is treating that, and using geometric optics as the approximation for the physics, going to give you a view of the world?
And then as people started to use other information — so another thing people use is, well, let's somehow look at how the Earth as a whole responds rather than this wiggle, and look at the normal modes of the Earth and normal mode frequencies of the Earth. They do that same thing for the Sun and stars. So again, that's this enormous data reduction from these time series measured in different spatial places, to try to turn it into a time series of spherical harmonic coefficients and then do Fourier analysis on the time series of spherical harmonics. And again, the goal is both data reduction, to get to an amount of data, and also to separate the response of the system from the stimulus.
Other data reduction problems on this kind of a massive scale that I've worked on, in geomagnetism — this is an entertaining story. The Danish Meteorological Institute flew a satellite called Ørsted, launched in 1999 with an expected 14-month mission. It's now been collecting data for 14 years instead of 14 months. But the community, when it first came out, wasn't equipped to deal with the data sampling rate that the satellite was producing: roughly 100 Hz data, and people wanted to deal with 1 Hz data. So how do you go about reducing the effective sampling rate without throwing away information, and perhaps gain some robustness in the process, so that gamma-ray bursts or this or that aren't going to totally trash your vector signal and affect what you're doing? So those are some of the kinds of things that I've been working on.
PÉREZ: Michael?
SILVER: So I work in the area of human cognitive neuroscience, and many people in our field are interested in, fundamentally, how the brain works — what sort of principles the brain uses to represent and process information. And in the last couple of decades, we've as a field developed a lot of measurement tools and experimental techniques to measure brain activity, getting very large data sets of brain activity over a wide range of spatial and temporal scales. And I feel like what we really need in our field is theoretical models and conceptual insights to be able to extract patterns from these data. And there's two general areas where we must work with other fields and obtain insights from other fields in order to be able to make progress in our questions.
And one is working with people who've studied perception and cognition. Psychologists and psychophysicists have characterized the basic principles of cognition and perception, and this gives us tremendous insight about what sorts of patterns to look for in the brain, and also really helps with experimental design. So unlike some of the other fields here, we are not relying on events like earthquakes and supernovae that happen outside of our control. We have a lot of experimental control and can intervene in the system, and we can use our insights about how perception and cognition work to design experiments to evoke very particular patterns of brain activity. And this allows us to get a handle on these really enormous data sets, and not just get lost in so much data, not knowing what sorts of patterns to focus in on. So that's one key component of working with others to analyze and work with big data sets.
And in the other, we just don't know the fundamental neural code — what algorithms are used by the brain to process information and to move information around to different areas. And so here it's very critical for us to work with statisticians, computer scientists, to have analytical tools and statistical measures to, again, be able to take these enormous data sets and identify patterns, test to see whether the brain uses these patterns or not. So one thing that's unique to our field is we're presented with this highly adapted, intricate, complex brain that evolution has been working on for a long time, and we're trying to unpack the history of evolution — how it's given us the brain that we have. And so we don't know a priori what the design principle should be or what the algorithm should be. And so it's very important for us to work with other fields to be able to gain computational insights that we can then apply and test to see to what extent they explain brain function and brain—
PÉREZ: Do you guys want to check your mics? I'm not sure — was everyone able to hear the last two speakers? Okay. Do we have the mic for the audience for later? Oh, yeah. Okay.
ALLEN: Okay, so I'm Richard Allen, and I'm a seismologist. And the first thing I want to comment is how exciting it is to hear so many talking about seismologists. We are in California, so there's a bias. So right there is one of the things that I find most exciting about this sort of effort on campus to get these connections going, because it allows us to exchange ideas. So that was the first comment I wanted to make.
STARK: So I just have to— right, my PhD is in geophysics, and my dissertation is in seismology.
ALLEN: But you're not a seismologist now, in a sense. That's what I meant. But the comment I want to make is, I'm going to add to the comments you just heard on seismology. You just heard about the problem in seismology of data reduction. I'm going to, following Josh's last slide, square the— because not only do we have to reduce the data significantly, and we've had to do that for a long time, the amount of data we're about to have is about to completely change.
So seismology uses typically hundreds of sensors that are distributed over a region. So we use a global seismic network that's a few hundred stations; the Berkeley Seismo Lab runs 70 geophysical observatories in Northern California. And so that gives you a sense of the sensors that we have distributed around the globe. And we can use that data because in fact it's an incredibly uniform data set, as these networks of very high-end, expensive equipment have built up over decades, and we have a very formalized and standardized process of gathering that data, collecting that data, archiving that data, which means that it's very easy for seismologists to then go and use it, because it's so uniform.
So that's about to change, because I'm guessing that almost everybody in the room has one of these in their pockets, and these all have accelerometers in them that can also record the ground shaking. So that means that just in this room, there's probably about twice as many sensors as the Berkeley Seismo Lab runs right now. So that's a good thing or a bad thing, depending on your perspective. The catch is that the noise— the quality of these sensors is terrible. And so the good news from the Berkeley Seismo Lab's perspective is that in fact the quality of what you would get from all of these sensors is significantly less than the single instrument that we have on a seismically isolated pier in the basement of Haviland, just a few hundred yards away.
So the challenge that means is: how do we really capture data from all of these sensors that are sat in the room right now? And that obviously means, first of all, the technical challenge in how you actually get the data from everybody's individual phones to a place where we can make use of it. And then how do we analyze it? And we have no idea — we being the seismology community — how to make use of that data. And so that's the real challenge, or one of the challenges, I think, in our field.
PÉREZ: And Josh, maybe you want to— after you've heard your panelists, is there anything that you'd like—
BLOOM: Well, one of the things to wrap up with— one of the ways that struck me about how I'm hearing the neuroscience question phrased is really almost theory-driven. You were saying you have a theory of how things work, and then you set up an experiment to test that theory. And I think that's different — although I'm not 100% sure — than the completely data-driven “something just happened, let me figure out what's going on.” Or I have a question, which is — the question doesn't have to have a theory behind it — what is this? If I push this part of the brain, does my eye twitch that way? You don't have to have a theory of why that happens to actually create some sort of causal inference that you wind up believing. And that's wonderful. I would love to be in that place.
But what I think of in astronomy — in some sense like earthquakes — is that we don't know when the next thing's going to happen, and all we can be is reactive. And the best thing that we can be as scientists, from a data collection perspective, is set up more and more sensors, with as best quality as we possibly can, to just try to capture when something happens, so that in the future we can go back — and maybe the future is over the next couple seconds or the next 10 years — and try to reconstruct something that we'll call physical, that will actually, after the fact, create a theory about why that actually is.
And so maybe— I was thinking this is fairly unique to astronomy, in the sense that the best we can do is just collect photons, right? The universe decides they're going to throw these photons at us, and it's silly for us to not put more detectors on them. Silly for seismologists not to have sensors all over the place, because unless you can go in and start drilling — well, I guess you could do fracking, right? But that's another panel discussion — you're not able to basically create experiments that test theories. Is that true?
ALLEN: No, that's true. It's all about having enough sensors out so that you capture the earthquake when it starts. People talk about— and also looking at things like the fault rupture processes themselves, where you can actually go and drill. That's the closest that you can— and you can drill through these things, but the cost there is just prohibitive. So we're forced to just be in this wait-look-and-see kind of approach.
STARK: So I see that there are things that are different that have to do with how rare the transient is. We're both talking about transient phenomena, and if you're talking about something that's rare, like a supernova, where you catch it at the right time, that's one story. But there's other kinds of transients: the sea pounding on the shore creates microseisms all the time, and you can use that to state— where the Sun is convecting turbulently, that's kicking it all the time, and you can observe the response to that. So again, you have some of this transient, not exactly stationary thing going on, but you get observations all the time, as opposed to observations every once in a while. And so then another distinction is: do you care about what's kicking the system, or do you care about the system's response to the kick, right?
SILVER: In neuroscience, there's also an emphasis on increasing the density of sensors and recording from as many neurons as possible, or increasing the spatial resolution of brain imaging to be able to have the smallest unit of tissue that you can. And I would argue, if you could record from every neuron in the brain and make that data set freely available to everyone in the world, we would not have any extra insight into how the brain works. The conceptual model is critical. And there's thousands of examples from perception science and cognitive psychology where a phenomenon has been characterized — an optical illusion — and then neuroscientists have found neural correlates of that in the brain. And I'm not sure there's a single example of just pure physiological observations predicting some perceptual, cognitive phenomenon, and then that turned out to be true. So all of the insights and ideas and concepts are going the other direction. Part of that is just that neuroscience is a very young field, and the recording methods are very young.
BLOOM: You're absolutely right that the concept is essential; you can't really get anywhere without a solid theoretical model. So I guess I had a question, more of a meta question, about different types of connections — not neuron connections, but the connections between the three of you, and the fact that you're asking questions of similar types of data sets, or you're dealing with similar volumes of data, certainly all in the time domain, at least. How did you find each other? Have you actually worked together? Your PhD was in seismology; you're a seismologist.
We have a significant issue on the Berkeley campus, in that everyone is awesome, and yet everyone tends to be fairly siloed. And it's not until we recognize that we have a need for somebody in, say, the stats department — I know there's a stats way to think about this, but I can't do it myself — until we articulate that need, I've found it pretty hard to just have the kind of water cooler effect where you wind up finding each other organically. And I suspect others maybe feel the same way here. It's an awesome place, and the people here are awesome, but it sometimes feels to me that we're lacking the connective tissue. And a bunch of us are thinking about how to just create those connections more organically. So maybe you could— I don't want to put you on the spot — have you ever talked about problems together, for instance?
ALLEN: So we had— I was on a committee for one of the students in the statistics department, but I think there could be a lot more. And this comes to— I wanted to comment on your last slide, the novelty squared challenge. That of course is a very exciting prospect, and I completely agree that's what we should be aiming for, but I don't think that that is all that we should be aiming for. I think that there are many problems where domain scientists need tools that people in the statistics department would find very mundane and very boring, but still we have to discover, right?
And so what we need to do is come up with some sort of framework where we can connect people with the tools they're looking for. Right now we could use tools for our analysis, we could use technical tools for our data collection, and we need to create a way that we can identify those, make use of those. And I think that that process would then lead to the novelty squared type problems, when you suddenly realize that you've run out of the kind of tool that you want. And so for me, as I think about this, I think of this being something like the role that the library has played for the last hundred years, right? When you want to go and discover something, you would go to the library. Well, we need that kind of infrastructure to be able to find tools and technologies.
STARK: So since we're among friends, including our friends on the internet who will be watching this later, I'll make some rude comments. I think that we really are at an interesting tipping point, because up until very, very recently, the attitude that I saw among physicists, cosmologists, astrophysicists was largely: first of all, axiom one, the physicists are the smartest people in the room. Axiom two, whatever comes up in a physics problem is physics. And so the consequence of this is there's really no reason for physicists to talk to anybody else about what happens in a physics problem. And so what's changing is there's a greater recognition that there are some themes that cut across more than one discipline, and some people who have expertise in that, and maybe it's worth talking to them. Yeah. So I guess I can get away with saying that partly because I'm some flavor of physicist, historically.
So I think it's a really interesting— In thinking about how to define data science, I came up with the idea that it's maybe a union of intersections. So if you have a tool that's useful in more than one data-oriented discipline, so that you have those tools that are in the intersection of two or more disciplines, and then the union of all such tools, right? That somehow, if it's useful only in one place, then maybe it really is just physics. But if it's useful in other places, then maybe it's data science. Maybe it—
PANELIST: So, yeah, some of my best friends are physicists, so I can also say that. This customer— yeah.
BLOOM: So I think you're absolutely right. We need you — we need your cohort, right? We need statisticians and computer scientists like never before. But I think the novelty squared statement is also that I believe that you need us as well, because we're creating real-world problems — although we're astronomers, not that real world — we're creating real-world problems that are novel. And you all need to start creating tools and frameworks and theories on your end that actually surface a novelty on our end. And I think you need us to basically give structure to what's important with the types of data we actually have.
STARK: So I completely agree. Let me elaborate a little bit. So there's a couple of different issues that come up in this. So first of all, I think good applied statistics is statistics in service of a real scientific problem. You can't do it in a vacuum. And there's an awful lot of methodology that's developed with the hope that perhaps someday there will be something that satisfies the assumptions of the methodology, right? And there's also a lot of misapplication of things, because you just say, “Well, it's in a book, it's got to be applicable.” So I absolutely think that the best applied statistics is done in service of a new scientific problem. That said, there are an awful lot of existing tools that could be put to good use in many applied disciplines. And I think we're seeing the same issues around the academic second-tier status of data science, in some sense, coming up in that direction, because an academic statistician who devotes his or her time to simply telling you what button to push in an existing software package is not going to get a lot of academic credit. It's not an—
PANELIST: Is there a sensor? Did you pick up that vibration?
STARK: So if we don't— it doesn't help to be the slave, or someone who amounts to being the programmer for the job. It doesn't help academically. So how to deal with that in such a way that science gets the best deal and the people who are helping also get a good deal?
PANELIST: I want to just do a quick—
PÉREZ: Yeah, well, I just wanted to make sure that we also open it now to the floor, so— but if you wanted to—
PANELIST: Well, let me do— let me just, for my own curiosity: how many in the audience would consider yourself a data scientist, by whatever definition? Okay. And how many of you would consider yourself a data-driven— Okay. How many of you would say that the work that you're doing is inherently— It's a pretty good mix. I think we have a biased sample here, certainly. So that was just my own—
PÉREZ: So we also wanted to make sure that we leave time. And as I said, there will be coffee and tea and cookies outside. But we have a great team here. I don't know— Michael, also, in trying to make sure that— we're four out of five physicists of some variety in the panel, and trying not to drive the stereotype any worse than we already have, I also wanted to give you a chance to comment before we open it to the floor, if you have anything else.
SILVER: What's that? On the question of the irrigation in the room?
PÉREZ: No, not on that question.
But no, I'm happy to take care of the audience. Yes, and so please — there, we have a microphone. So if you have any questions for our panelists, please, and if you have suggestions of how to stimulate data-driven science on campus. Yeah, so for those of you that raised your hand: how did you find somebody who is your counterpart on the other side of that discipline? And please, when you ask a question, do tell us who you are. Part of what we're trying to do here, as I said, is build community. So don't simply ask a question; tell us who you are on campus and what department you are in, so that we can find you later. Oh, well — hold on, Fritz.
AUDIENCE: Hello, I'm Shannon Neulon. I'm actually visiting from Stanford. It's a question for Michael. I found your comment very useful about, essentially, if they got the output from every neuron in the brain and put it together in the data set, we would discover perhaps not as much as we think we would discover. What, therefore, do you think of, say, the Henry Markram project that the European Union has just invested a billion euros in? What do you think of similar projects that attempt to do the same, but on the basis of fMRI locations? And thirdly, at a rather comic level, since the neuroscience–physics thing has come up: what did you think of the argument that Sheldon had with Amy about whether neuroscience could be reduced to physics or vice versa, in The Big Bang Theory?
SILVER: Wow. Well, for better or worse, I'm more qualified to talk about the first two data science initiatives; I haven't seen the TV show. So I think — as a way of background, the Henry Markram initiative that's going on in Europe, in a sense, I think, as I understand it, is trying to simulate a very small portion of cerebral cortex based on all of the synaptic connectivity and physiological responses of neurons, simulated neurons. And it's far from my area of working with humans and non-invasive measurements, but I think that kind of detailed measurement will ultimately be critical for testing models. I'm not sure that the strength of our models and the fundamental principles with which to build a simulation is really there yet, but I think the field is so young, it's premature to really close off any areas. And so those kinds of data will absolutely be critical at some point. Whether they will prove to be useful in the short term, until we catch up with a more sound anatomical- and physiological-based model of what neurons are doing, I think remains to be seen.
And there's a parallel initiative that was announced recently by the Obama administration, and it's a very similar kind of question: what we really need to do is now take these methods and record from as many sites as possible, and then somehow the answers to brain function are going to spontaneously emerge from these large data sets with these very finely spaced sensors. And so I'm not against collecting those data; they will ultimately be very helpful, but they may be in stasis for a while until the theoretical side catches up.
PÉREZ: And again, please tell us who you are, for those of you…
AUDIENCE: Yeah, so I'm Fritz Sommer. I'm from the Redwood Center for Theoretical Neuroscience here. And so I just want to address also the question: how do we go about establishing data science here on campus? And I think one interesting thing is really that data — it sounds a little bit dry, “data science,” and also if we then say, well, people out of frustration will turn to it. But I think what is really very exciting is that it's a common currency that now really brings together methods-driven research and really problem-driven research.
And an interesting way, I think, also to establish it is to popularize the problems that people have in the field. So, for example, at the Redwood Center we are holding every summer a two-week summer course for students that are quantitatively trained and have usually not too much idea about what neuroscience is, and we explain to them the problems — of course also the methods that we use right now — but what we hope is that we can actually bring interest into these other domains, and make them think more broadly about science and start to think about neuroscience. And I think this is also really an opportunity we should use: not only trying to popularize our methods that we have as statisticians, but also think, if we have an interesting science problem, how can we actually communicate that into other communities?
PÉREZ: And can I briefly ask you, before you turn over the mic: what kind of mix of students do you get? Because I imagine you guys have background data. Is it mostly people who are going to do neuroscience, or do you get a healthy mix of disciplines?
AUDIENCE: I would say it's about 50% of people who already decided that they want to get themselves into neuroscience. Some of them already are PhD students in the neuroscience lab but want to tackle a problem that requires a lot of quantitative methods. And the others, they just come from physics, computer science, statistics, and they are just somehow excited; they see this as an area where they can apply their skills and interests.
PÉREZ: Thank you. I think we have one question over there. Okay, if we could send the microphone over — and again, please let us know who you are.
AUDIENCE: Hi, I'm Simona Zompi. I'm the director of the Center for Global Public Health, and we are right now reviewing a new program, a master of science in global health, and I would really like to add some data science in there, because I think this is the next — for the next generation, that's where the jobs will be. And so I'm wondering how can we integrate that, to which level, and to which schools I should look into to try to integrate this into the global health MS.
BLOOM: One thing I'll just react to immediately — I don't have an answer to you, actually — is that what I've been surprised by, what I should stop being surprised by, is that when we talk about data science and we talk about how we connect with each other, it's inevitable that fundamentally what we wind up talking about is education, right, and about training. That's where all of this great science starts: with the next generation of students. And it's come up, right? We've just now brought it up. And so I think it is as fundamental as thinking about building communities of established researchers; we also want to be thinking about building those communities amongst the students, and amongst a population that doesn't know any better yet that you have to be siloed and you have to tag yourself with “I do this, this, and this” so that you can get a job in a certain department. At some level, working with students who are interested in solving problems and bringing whatever toolkits are needed to bear on the problem to do it — that's where I think we have a chance for a really great impact. So again, I didn't answer your question.
STARK: This is perhaps germane to the question. I've been talking to Josh and Fernando and Aaron Culich about coming up with an upper-division course in reproducible and collaborative data science that would be cross-listed in a handful of departments, such as computer science, statistics, astronomy, and I don't know where else. But maybe this is an opportunity to have something that your students could take, depending on how many of them there are. Or — this is going to be in the fall, right? We're doing this as an upper division, listed in CS and statistics, this fall, but that's on a pilot basis, to try to develop something that's more overarching. It might even be a two-semester sequence: just an introduction to a variety of tools that people find in working collaboratively and with data, especially large-scale data. So good hygiene around coding, and this and that, and computational tools that come up in a broad variety of disciplines, so forth and so on.
I'm a little bit skeptical of data science as a thing that is taught somehow as a subject on its own, in the same way that I'm a little bit skeptical about statistics as an undergraduate major. If you want to go on and do really good work in something, you really need substantive knowledge in some underlying field, and the ability to go out and figure out what tools are useful to you. And I think a better background is somehow not some particular toolkit, but ways of thinking, and something you can actually dig your teeth into.
PÉREZ: I think we had one more question. Maybe we'll wrap it up there with you, so that I don't keep people from their lunches, but also from coffee and cookies. Please.
AUDIENCE: Well, it's quite related, actually. So I'm a PhD candidate with a background in robotics and computer science, but I actually work in public health and with atmospheric scientists who use low-cost sensors. Not all training really occurs in the — so since this is about time series, I'll phrase it this way: how are your changing views on this topic and the relevance of data science — how are they reflected in changes in the way that you train your graduate students?
BLOOM: So I'll react to that. What I saw — I did a poll, basically, of all the things that I required my graduate students to be good at, or at least know a lot about. And it was a whole bunch of different types of databases, it was a whole bunch of programming languages, it was domain knowledge. And I know I wasn't taught that way. And so with my students at the graduate level, a large amount of their training is certainly not happening in the classroom. It's saying, “Now we need to talk about how we're going to actually store all this data and move it around on the network and do parallel computing.” Some of it's by osmosis; some of it's directed.
But the requirements to do the things that I do in my lab and in my work are quite — and one of the reasons why I started this Python boot camp with Fernando and company is the recognition that I was asking people to come into my lab who knew Python, and it turns out no one in astronomy was being trained in Python, and they didn't have time as an undergrad to take a bunch of computer science classes. And so part of the reason for the creation of the boot camp was essentially to better put my money where my mouth is — or my time where my mouth is; that's the same thing, actually. And we started that as essentially a labor of love, right? So we don't get teaching credit for this, but we're expecting something like 300 students in the fall for this, and then we jump into a seminar class to teach people how to be data-driven scientists in the Python language, just because I need my students to know how to do that.
So one of the great things about being at a university and trying to do research is you wind up figuring out where the deficiencies are at the undergraduate level, when you wind up seeing a cohort of people coming in not with the skill sets that you think that they need. And then you can just create classes that get to them early — not the same people, but their equivalent. So it's had an impact on the way that I teach and what I teach.
PANELIST: So do we have — yeah, if we have any quick last words from you guys, by all means.
SILVER: I thought it was very interesting, your comments about education and statistics — just, if I understood correctly, that sometimes you can get a very solid knowledge of statistics, but it's important to be able to have a concrete field to be able to apply it to, for statistics to be truly useful.
STARK: And I would certainly rather see an applicant to our PhD program who was either a math major or a physicist than somebody who was an undergraduate statistics major.
SILVER: So in a sense it's the opposite in part of the biological sciences, where there's a lot of very concrete knowledge, not much in the way of unifying principles in biology. And so people learn a great deal about the specifics of the system or the organism, and that's a big part of our curriculum — classroom curriculum. And then for research projects, they're often very siloed in very particular model systems. And a lot of our students — this includes myself and peers as faculty members — we've learned as we went along to solve particular problems, and learn programming that we need to do to get the job done, and we don't really have a broad foundation, and we don't really know all those areas that we've not been exposed to that would be very useful for our problem. And I think we would all benefit from having much more formal education in statistics and computing that's part of a graduate curriculum in neuroscience or biology as well, and maybe less of the actual facts about the particulars.
STARK: I didn't mean to imply that learning some statistics as an undergraduate was not a good thing to do. It was more the idea of majoring in it.
PANELIST: Subliminal message to reduce the pressure on the major. But I see how many students in our program have been attracted to the Python boot camp and other programs that you guys have organized. It seems obvious that it's filling a need that's not currently being addressed by the curricula as they are.
ALLEN: If I can amplify this, because I think this is key, and in answer to the question: I think that we require our students to really have the domain knowledge in seismology and geophysics, and I think that that's really important, and I don't think that should ever — and I don't think that will change. But at the same time, they're then left to pick up these skills, as Josh was saying, largely on their own. And so how do we really facilitate a progression? It's by creating these kinds of opportunities for them to plug into these tools that they need. Boot camp is a great example, but there could be so many more that we could do on campus, and I think that would make a huge difference to what we're able to do.
STARK: Here's a plea, perhaps to the VCRO — I'm not sure exactly — but I would love to teach a graduate-level introduction to statistics for physical science, or for science more generally. The problem is we don't have the FTE. If I'm teaching that, I'm not teaching something else. But it could be a really, really good thing to do. I'm on enough physics PhD committees, astronomy PhD committees, computer science or electrical engineering PhD committees, the occasional earth science piece — there's simple stuff that could really improve things, but they're not getting a chance to learn it.
BLOOM: So, just want to react to that. What I wind up hearing is: you say we've got this boot camp, and you're starting a class, and you'd like to have one FTE. What's pretty clear, I think, is emerging is that we need to have a formal structure on campus. And I don't know what that is yet, but I'm starting to see that this shouldn't be done piecemeal — “I just have to create this boot camp, because otherwise I'm not really living up to my educational mission.” I think if we were able to wrap around it and say the job of this entity is to do all of these things in a coherent way, where it's not just one boot camp and another one, but they're actually all connected to each other, and it's not just one course at the undergraduate level, another at a graduate level, but an entire curriculum of how to do this — and not necessarily in its own right, but maybe in concert with other curricula. So I think what we're hearing is this burbling need for a larger structure on campus. And I think we'll wrap it up with you, so that…
AUDIENCE: Okay. I'm Matthew Bailes from Swinburne University in Australia. Do you think one of the problems is that Berkeley encourages only new academics who are going to be — and you don't have anybody who's a really good scientist but doesn't want to be on the front cover of Nature, or — and doesn't want to do work for somebody else to make their career better, but they want to be the PI on everything? And that you'd be better off having a more diverse section of academics you're hiring, some of whom are really smart, but they don't have the egos of some of the people on stage, perhaps. There's a great Gary Larson cartoon: “It's another case of too many scientists and not enough hunchbacks.”
PANELIST: It's probably a perfect opportunity — why don't you take that one?
PÉREZ: Yeah, perfect. But — perfect — as the one member here of this panel who is not tenured faculty on campus, I would like our university to react to that in a positive way. Let's put it that way. And who's not on the cover of Science, or — but I don't want to keep you people from your lunch, from tea and coffee and cookies.
But I do want to thank Kaya and Cassie and the VCRO for getting this effort started. This is the first one; we're alpha testing this thing. Please let us know how this series can be made better. Help us. The mailing list, datascience@calmail.berkeley.edu, is how we're broadcasting this, so you can let us know if you think you have a great speaker in mind, a topic that needs to be addressed, somebody who maybe is visiting you from out of town and would be a perfect person, who's in town for a few days. We're trying to have these once a month, regularly, for the foreseeable future. So hopefully this is the start of a useful community and a useful discussion on campus around things that are both instrumental to the university but also of very broad scientific impact — I think that is clear. So thanks, everyone, for coming, and thank you to our panelists and speakers.