Data literacy before big-data proficiency: what Berkeley's Python boot camps and graduate seminar showed about training domain scientists, and the open questions — where such courses fit in the curriculum, who teaches them and how they're credited, the ‘novelty-squared’ problem, and inclusion.
Given April 11, 2014, as the first talk in the workshop's session on courses, curricula, and interdisciplinary programs (chaired by James Frew, with talks by Peter Fox and Bill Howe). The National Academies Press proceedings summarize it in Chapter 4.
“Just because you learn French doesn't mean you're a good poet. I think Python is this modern gateway to computational computing, and it's the art and the poetry that is enabled by it that we really, really want to get to.” – Joshua Bloom
“If you add ‘data science’ or ‘big data’ to your course name, you increase enrollment tenfold.” – Joshua Bloom
“What I advocate is that, in terms of training, even at the higher-ed level, we ask how can we get our students more data literate before we start thinking about how can we train them in big data proficiency.” – Joshua Bloom
“You're somebody who is a fabulous teacher of a programming language. That would be a hard person to land in a computer science department, and that'd be a hard person to give tenure to in a biophysics department. So where do we find a home for these people?” – Joshua Bloom
“The last thing I want to have happen with an interdisciplinary collaboration is that my CS and stats colleagues find their contribution routine, if not mundane.” – Joshua Bloom
FREW: …person I know that actually has eScience in his job title. His specialty is in giving scientists access to big data, and he'll be talking to us about the Introduction to Data Science MOOC, a massive online course that he developed and taught for Coursera. So he's also the only person I know who's taught data science to more than 100,000 people at a time.
The format we're going to follow here is each speaker is going to be given 30 minutes to talk, which is to say I'll give them an indicator at 30 minutes, and they'll probably talk a little longer, but not too much. After that, I will solicit questions with respect to the speaker's remarks. If you have a more generic question that would be better answered by the panel of all speakers, I would request that you hold it until the final speaker has finished.
If you're watching the webcast, you may, as you've probably already figured out, submit questions in the box below the video frame. If you do, please give us your name and affiliation, and the questions will be cherry-picked and relayed to the workshop by the folks here who are helping us run it. All right. So, Josh, if you'd like to lead off.
BLOOM: Well, thanks for the introduction, and thank you to the organizers for the chance to speak today. I'm particularly thankful to the previous two speakers, who I think really teed up this talk, hopefully pretty well. I should say that the explicit goal of training our undergraduates and graduate students is to prepare them for a career in forefront science. And if some of them go off and don't do forefront science, we certainly hope that one of the residual effects is that we're training them to become great data scientists, and empower that path that they take.
So for me, this fundamentally means arming these people with toolkits — not just training them on software and hardware frameworks, but methodological and collaborative approaches that span the full range of hypothesis- and data-driven inquiry. And it's a pretty bewilderingly large and increasingly large zoo of the types of subject matters that we ask our domain scientists to get good at.
Now obviously domain training is key, and so if you're a physicist, perhaps you have to get trained in laboratory techniques, or perhaps even the software and the hardware tools of your trade. And what should be clear, but we'll state explicitly here, is that the mentor-mentoree relationship that's always lived in academia assumes that all the other tools that are needed just somehow flow to you in some magical way — that you find all the time, between when you're not sleeping and not working, to pick up some of these tools that one day maybe become absolutely seminal to your own work. So you have to trade something off if you're going to be picking up these other parts of the toolbox. And this obviously makes things very hard. And it makes them hard not just because we need to learn them at a cursory level; it's because, as we heard in the morning session, we're also asking these people to get very good at some of these different toolkits.
So I did a little survey of the students in my group as we were doing some astrophysics projects — a data-intensive project on streaming data — and I asked them to write down all the different jargony terms that we use, in terms of just software infrastructure and statistical infrastructure, from one week. And this is what we came up with. And it's kind of shocking if you think about the fact that an astrophysicist, as an undergraduate and then as a graduate student, essentially has no formal training in computer languages, in database architectures and database installations, let alone web frameworks and beyond.
And so I think it's not just me in realizing that if we're going to be extracting value from an increasingly large amount of data, we need to get our students in the domain sciences capable and conversant, at least in some subset of these tools. So the question, I guess, we have here is: are we reaching a crisis mode when the practical need for innovation is just too hard to come by in the few years of higher education?
And again, we really can't hide from this in astronomy. We are facing a massive data deluge. What you see here is the 800-pound gorilla in the room for many of us, which is the Large Synoptic Survey Telescope, arriving in just a few years from now with a huge amount of data, more than most astronomers have ever had to deal with in the past. It's an NSF-funded project, and just to give you a couple of highlights of what it is that we're expecting: something like 800 million sources in the sky will be observed every three days. And so if we're updating our understanding of those sources, we need to have a fairly large computational framework to do this. We're also doing anomaly discovery and discovery of the known unknowns, millions of supernovae a year, just completely dwarfing most of the data that any of us have dealt with — talking about something like 20 terabytes a night.
And then there are other projects on the horizon, and actually already among us, in the radio bands: there's LOFAR and SKA, again with major U.S. national involvement. And we're talking about computational requirements in the hundreds of petaflops to be able to just keep up with the data flow. And on and on and on.
The nice thing is we've been able to cut our teeth on the precursors to many of these different projects with existing surveys. And one of my ways in which I come to this problem of teaching Python to data scientists is the recognition that our ambition really is to get rid of people in the real-time loop. And so I've been working for a good part of the last several years on automating the scientific stack for time-domain discovery around so-called transients. And we're really trying to take everything from planning observations all the way to published work into a regime where you've got really streaming workflows, using machine learning that approximates human insight for doing discovery, and then actually follow up with robotic telescope resources. Some of this is embarrassingly parallel, so scaling is sometimes trivial, but in many cases it's not. And so we do have some large bottlenecks when it comes to just dealing with the raw data.
Now, with the NSF's help, I've been able, through a couple of grants — CDI and Big Data in particular — to grow a group of interdisciplinary scientists from computer science and statistics that allowed us to do a couple of different things with this new workflow. So I'll just point to a couple of successes here and then talk about how we can start training our students to follow on such successes with their own toolkits.
So we started building out these capabilities in my group, and we wound up getting basically a real-time framework that allowed us to find transients in the sky with the so-called Palomar Transient Factory data. And that's led to about 50 refereed journal articles, basically making use of a real-time machine-learning framework that would essentially act as surrogates to what used to be essentially grad students looking at lots and lots of images. And so in addition, we built probabilistic event classification catalogs and started to do some amount of innovation on the statistics and ML side, where we started coming up with some new approaches to active learning.
But as I was building this capability, I started to think about where we needed to start sending our students as they would arrive into the group, quite green in many of these ways, in computing and machine learning. Where would they need to go to get the practical knowledge just to get up and running and be able to keep up with everyone?
And so Berkeley obviously has a great history with parallel computing, and there are a lot of different resources on campus, both at the course level — those are generally geared towards computer science students and statistics and math students — but then there's also, fairly recently, this growth of so-called boot camps: these two to three intensive training sessions, almost vocational at some level, to get people up to speed and using these fairly well-known frameworks now. So BDAS, for instance, from the AMPLab is a good example: Spark and Shark, et cetera, are next-generation MapReduce tools that seem to be gaining a lot of favor in the big data world. There are also boot camps from the Parallel Computing Lab, and this is getting people up to speed on all the different concepts and then actually putting into practice a bunch of different parallel computing paradigms. And then last — this is fairly new — the Information School at Berkeley started an entire degree program around data science.
The problem, though, is that all of these courses are taught either by stats or computer science professors, and almost exclusively they're aimed at engineers and programmers that are heading towards industry. And this is not to say that computer science professors and statisticians can't be teaching domain scientists; it's just that this hasn't been what's gone on in the past. The question is, is this program — the data science program, it's a master's program put on by the I School.
And so I started wondering: well, what about me? I had been working with Python for the last 10 years and recognized how useful it was in my own research life, and thought, well, maybe I should start teaching some domain scientists how to do some computing, where they could start off very small, but then realize that they could take that computing to scale.
So I started these Python boot camps at Berkeley in 2010 and got a fairly good reception. I was overwhelmed, even with just the 85 people that showed up, by just how many would sit through essentially nine hours a day. That room was very hot — that was basically at the very end of summer, and you see all the water that's lined up on the table there. But it was fairly grueling: there would be homeworks at night, very intensive sessions of teaching, and then a lot of hands-on, with a lot of camp counselors who would go around and help people in real time.
And the only prerequisite that I had for that boot camp was that you had to profess that you knew a computing language. So I'd ask people in a questionnaire, “What languages do you know?” And if they clicked “none” — they didn't know it — but I threw them off the list. If they clicked “Excel,” I would call them up and say, “Hey, let's talk about Excel not actually being a computing language.” But I really didn't want to have to teach computer science concepts; there's not enough time to do that. I wanted to teach the language itself, because it's a beautiful language and it's becoming more and more widely used.
So this got some traction. We got a bigger venue, a nicer venue, and kept going with it.
Python's been mentioned a few times already in this workshop, but I thought, for those that haven't heard of it, I would just tell you my thoughts of it being a modern superglue language for science. It is not the kind of language you would do hardcore, very large-scale computing in natively, but as a hypervisor over all of that, it's not a bad idea. It's a high-level scripting language. It's been called executable pseudocode, so it's actually quite easy to read. It is open source, therefore free, and it's got a huge and growing community in academia and industry behind it. I won't go into the details of all the different reasons why it's a very interesting language, but suffice it to say, it's not just got built-in fast computational cores that connect to things like LAPACK, but it's also got extensive interfaces to third-party frameworks, and I'll talk more about that later on. So for me, it was emerging many years ago as this lingua franca for scientists.
The only problem is it kept growing and growing and growing, and we started getting bigger than any room that we could get access to could hold us. And we wound up realizing that this is something that we should be pushing out further, beyond the confines of Berkeley. So we've now gotten to the point of live streaming and archiving all the lectures. We've open-sourced all the material on GitHub, and it's now being widely used and repeated. There was a session at Goddard Space Flight Center nearby — it's a NASA center — that basically redid the boot camp without essentially any of my knowledge. I got an email saying, “Hey, we reused it and we credited you. Is that okay?” I said, “Awesome.”
And one of the challenges here is that this is not an inexpensive thing to do. It costs about $18K to put on one of these things. But we've had some nice support from our Vice Chancellor for Research, and my Big Data grant also helped out on that.
And the important thing about Python is — the way I talk about it is that just because you learn French doesn't mean you're a good poet. I think Python is this modern gateway to computational computing, and it's the art and the poetry that is enabled by it that we really, really want to get to. So in some sense, the boot camp became a prerequisite to something bigger, and perhaps something deeper. And for me, that was a follow-on course, a seminar course that we taught primarily for graduate students, and we called it Python Computing for Science. And this also gained some traction over the years.
This became part of a degree program at Berkeley — or I should call it an ancillary degree program — called the Designated Emphasis in Computational Science and Engineering, allowing people who are getting a PhD, say, in biophysics, to get this extra recognition — essentially a minor for PhDs — that they've actually done extra coursework in computation.
So we're now teaching basically each week one of the many concepts that you see up here. And obviously we can only scratch the surface, but we start off each week really just with the extreme basics: what is a database? Why do you need a third-party database? Why can't you do this in pure Python? Et cetera. And then go on and have them, by the end of the week, be able to accomplish a fairly large database program and some functionality with databases in their homework. And so when we do parallelism, we start off with multicore parallelism and then multicomputer parallelism, and then we have just basically a few hours talking about MPI and GPU programming. So we're really only scratching the surface, but we are trying to get some level of depth with the homeworks.
And it's been really exciting. We had essentially four or five people take this course for credit, but a number of postdocs sitting in on it. And what I'm really excited about is that we asked them, essentially, at the end, to create these final projects that, with Python as the glue, basically connect to research tools that they were already using and make them better — perhaps create a visualization around a command-line tool they were using, or do something that they'd always had on their mind that their research group would absolutely love to have, and just build it. And then I would give them credit for it. And so this is just a smattering of some of the titles of the final projects from December.
But one of the things that I wanted to point out here, really, too — and I'll get back to the gender ratio towards the end of the talk — is just the breadth of the different disciplines of PhD students that came to this course. This was the largest astronomy graduate course ever taught at Berkeley, and you see there were four astronomy students. It's because there was some unmet need throughout our graduate programs. And it's not just physical science; we had people in social sciences as well. And so they were seeing the need for something that wasn't being well served.
But obviously I came to the table with my own interests, and it was wonderful to see people build essentially their own research projects. And this was something that would take a few weeks, and they'd get credit for it. But one of the most heartening things was that one of the graduates of the boot camp, and then graduates of the seminar class, went on to write probably one of the more important papers in astronomy in the last few years. This just came out; it was in PNAS. And this was making a fairly strong set of statistical statements about the prevalence of Earth-sized planets around Sun-like stars in the habitable zone. This is a tremendously difficult set of measurements to make using Kepler data. You see the noisy data, and ones with lots of secular variations on long timescales, at the top, and then what they were able to do is denoise this. And this is a fairly large computational framework, from somebody that had started a few years earlier just basically knowing some IDL and maybe some C, and going to the point of being able to do large-scale computations on supercomputers at NERSC using Python — that was great.
So because we're in D.C., I thought I'd have a quote from a president, from a science and technology speech. He said — this is Clinton — “Are we alone in the universe? What makes up the missing mass of the universe? And maybe the biggest question of all: how in the wide world can you add $3 billion in market capitalization simply by adding ‘.com’ to the end of the name?” Now, those that know their history will know that a few months later the stock market crashed, et cetera, and that was the end of the dot-com boom. So for me, I'm taking a lesson from this, and what I'll say is: if you add “data science” or “big data” to your course name, you increase enrollment tenfold. Remember, my original Python course was called Python for Science. I added the word “data” in it, and I wound up getting not a few students who were taking it for credit, but now about 40 students taking it for credit. And we also overflowed the system, and actually literally couldn't find a place on campus that could house us all. You see people sitting on the floor, et cetera. So again, we had something going on here.
So that's the good stuff. But here's a little bit of the dirty laundry. One of the questions that we're all asking ourselves at Berkeley — and I'm sure this is happening in many places where people are starting to work on boot camps and seminars and special lecture series and after-hours hackathons to get people up to speed — is: where does this really fit into the traditional domain science curricula? I have not, by any stretch, created a fundamentally different curriculum that people who are doing a PhD can take and come out and call themselves data scientists. I've essentially only instrumented two courses.
The problem is — and I think it should be clear to everyone — that it's often seen at the PI level that formal coursework competes with the research obligations for graduate students. And this is something that's always going to wind up being a tension. And as we ask people to build more and more tools and be more and more familiar with toolkits, it's going to become even harder.
And one of the other sneaky questions here is: is this actually too vocational, or too practical, to be teaching at higher education levels? If we're teaching somebody how to muck in on the command line and bang out a cool plot, are we really going into the deep concepts behind visualization and behind OSes? And is that actually something that we should feel bad about if we don't do? And I certainly don't have an answer to that. I just know that we have to figure out a way within higher education to fit this in.
And — not that this is a huge deal — but we don't often get credit as educators for actually providing these boot camps and running these boot camps. And one of the things that we are going to have to start asking ourselves is how do we create career paths, even through academia, where it might be that you're not a forefront researcher, or you're not teaching a graduate course in your specific domain, but you're somebody who is a fabulous teacher of a programming language. That would be a hard person to land in a computer science department, and that'd be a hard person to give tenure to in a biophysics department. So where do we find a home for these people?
It should be clear that a lot of the toolkits that we're having our students get familiar with and build up are not just big data toolkits, right? This is thinking about data in a very fundamental way. It echoes a lot of the different things that we heard earlier today. And for me, getting people to know how to drink from the data water fountain before we shoot the hose at them is an absolutely critical one. And so what I advocate is that, in terms of training, even at the higher-ed level, we ask how can we get our students more data literate before we start thinking about how can we train them in big data proficiency.
And there are some really basic things here when I talk about data literacy, and I obviously won't have time to go into all the different facets of it, but statistical inference is one. You'd probably be shocked, if you give somebody a couple of data points in an x-y plane and ask them to fit a line to it, at the different tools and the different approaches that they'd go after that with. There's quite an amazing treatise on fitting a straight line through data — that's a 55-page treatise written by an astronomer and statistician, David Hogg et al. — and I've made a link to it here. This is the kind of thing where it will blow your mind to think about all the nuances of how you think about throwing out different parts of data. How do you do that in a statistically rigorous way? How do you make predictions about data that you haven't seen? How do you do this in the context of censored data? And there are all these different aspects — and this is not big data; this is 15 data points — that get into some really interesting questions about how much we can possibly teach our domain scientists in the limited time that we have with them.
And the other thing that's coming to the fore for me is around versioning and reproducibility. There's this great quote from Science magazine from this year saying, “Recently, the scientific community was shaken by reports that a troubling proportion of peer-reviewed preclinical studies are not reproducible.” And this has obviously gotten the attention of agencies, who are becoming more and more clear that they're going to wind up mandating some level of reproducibility around what we do. And reproducibility is coupled with versioning, but obviously it's not the same thing. Just because I version my code doesn't mean that I've created an entire environment where people can create reproducible science results.
So Git has emerged for me and many of my colleagues as the de facto versioning tool. And one of the new initiatives, headed by Philip Stark — he's the chair of the stats department at Berkeley — is the Berkeley Common Environment software stack. And he's also starting to teach a class called Reproducible and Collaborative Statistical Data Science. The idea is that they would essentially have these large virtual machines that anybody could wind up cloning and adding to, et cetera, but it'd be the kind of thing where I could write a paper and just say, “Here's the link to my virtual machine; you press this button, and you can reproduce all the results from the raw data all the way down to the plots from the paper.” And one of the things that I think is an unfinished question is how we actually do versioning of data, especially when we start to get into large data volumes. That's not a solved problem.
Now, in terms of versioning and reproducibility, one of the things that's really wonderful from my perspective is that there's also this confluence with these ideas and the Python environment. In particular, there's a fairly new initiative called the IPython Notebook, created by one of my colleagues at Berkeley, Fernando Pérez. And here we're able to basically do what people have been doing for a long time, say, in MATLAB — essentially do reproducible science in the browser.
So let's see if I can click this. There we go. And so what you see is essentially LaTeX, which automatically renders when you execute the cell. And this is essentially a nonlinear differential equation problem. We can change a couple of the different parameters here and solve these dynamically, and do visualization directly in the browser. And these are notebooks that I can send to you; I can share with you. And now there's this ability to actually interact with the visualizations in real time in the browser, where every time you do another mouse click, there's a whole set of computations that are happening. And this is happening just on a single core, on my laptop, but you can very easily extend this out to a multicore, even multicomputer, environment. So you can control, from just one of these browsers, a huge amount of computation under the hood. And I know that IBM Watson is using the IPython Notebook, for instance, to wrap a lot of their infrastructure and visualize it.
And just to say that I'm not completely proselytizing only for Python: one of the nice things is that the IPython Notebook is pretty agnostic to the language behind it. So you type in a couple of lines at the top of one of these cells, and now you have access to R functions, you have access to Julia functions, et cetera.
So in the last couple of minutes that I have remaining, I wanted to talk a bit about environments, and talk about how we create this next-generation data scientist, or domain scientist, who has all these skills. And since it hasn't been shown yet, I'll show you Drew Conway's now-famous diagram about what a data scientist is: this intersection, as we've talked about, of hacking skills, deep knowledge of math and statistics, and then all the domain expertise. And while, from the first speaker of the day, we had the notion that this is the kind of thing perhaps that Amazon might be looking for in an individual, I think what we also heard from the previous speaker is that the idea that we're going to have extreme experts in one person, in essentially all three of these diagrams, is very hard. This is essentially the definition of a null set. These are a bunch of unicorns that are overlapping, right?
And so the idea is that the way that we push forward — if we all agree that these are the set of skills that are needed to make progress on the industry side, or to make progress with real data problems in science — is that we should probably think about creating aggregations of people: that is, collaborations among substantive experts, math and stats experts, and then people who know how to code and build scalable infrastructures. It would be great if everybody knew some bit of everything else, but to ask them to be extreme experts, I think, is too much.
And then when it comes to actually doing research, there is this thing I call the novelty-squared problem. This is this extra burden that we have as forefront scientists trying to use toolkits. It's this idea that if I'm going to be pushing the envelope, say, in astronomy, and I'm going to be doing something completely novel and find, say, a new type of object in the sky, I actually feel a little bit bad going to a CS or stats professor and saying, “Hey, collaborate with me,” and all they say is, “Well, look, that's just a Spark cluster, and you've got to put it on an Amazon thing — do this and this and this.” Because for me, what is novel on the domain science may not be novel from a methodological perspective. And the last thing I want to have happen with an interdisciplinary collaboration is that my CS and stats colleagues find their contribution routine, if not mundane.
And so it's novelty squared in the sense that, to get fully engaged in that collaborative work, you need to find problems that are novel in the domains and also in the methodologies. And the only way that I can see that we solve this problem is we start speaking each other's language and start learning what are the forefront questions in these different fields.
And what has happened at Berkeley recently is, along with UW and NYU, we got a grant from the Moore and the Sloan Foundations to create such an environment — to create this ecosystem where people on the methodological side can talk to domain scientists and hopefully find ways for them to collaborate in novel ways. And the goal there is, again, just to create this rich ecosystem for domain scientists to collaborate with methodologists. And we feel very strongly that this is not going to be something that's going to be done virtually. It's going to be done with an actual space on campus, where people can come and actually find their counterparts who were looking for a novel question to answer.
We kicked this off in December of last year. I ran this Data Science Faire, and we had an overwhelming response from people across campus who fancied the work they're doing is interdisciplinary around data, and it was really wonderful. And these are two of my students from the Python seminar showing off some of their work — obviously very real excitement here.
One of the things that I haven't mentioned, when it comes down to creating environments for people to do novel work, is creating inclusive environments — that's expanding participation among underrepresented groups. Now, one of the amazing things for me is that when we ran our Python boot camp, about a third of the boot camp were women, and it was a similar number for our seminar. And then literally the day after my boot camp ended, I decided to put myself through some torture, and I went to the AMPLab camp, which was teaching Spark and Shark and the BDAS stack to a room of, say, 300 people or so, mostly from industry — Google, LinkedIn, Netflix. And I think I counted on one hand the number of women in that room.
So for me, there was this fundamental disconnect between the notion of whatever we're teaching people when we talk about big data and the kinds of toolkits that a larger group of people are actually looking to learn. And also, to note, there are only two women out of 22 speakers here in this workshop. So there is a problem. I certainly don't have the answers, but recognizing that there is a problem, and recognizing that there's also this pent-up need for learning data science skills, is absolutely a critical start, I believe. So I'll end just putting my summary slides up there, and hopefully we can take a couple of questions, either now or at the end. Thank you.