Benjamin Dean public lecture in the Academy's Morrison Planetarium: seen with enough sensitivity, nearly everything in the sky varies; surveys like the Palomar Transient Factory and LSST are outrunning human inspection; and automated discovery and classification are changing what it means to be an astronomer.
A Monday-evening Benjamin Dean lecture under the Morrison Planetarium's dome, followed by half an hour of audience questions. The Academy's audio recording (1 hr 38 min, from its former iTunes U collection) is linked above; no slides survive.
“So if you have a problem with Pluto being kicked out of the planethood, this is the movie to blame, and these are the people — Mike Brown and company — to blame for this.” – Joshua Bloom
“Instead of taking pictures of the sky, what we're really doing, and what we're really interested in, is taking movies of the sky. This is celestial cinematography.” – Joshua Bloom
“So one of the challenges upcoming is how do we get smart telescopes to talk to each other and federate themselves to do interesting science while the humans that own them are actually asleep. I'm sorry for those of you that are having images of the Terminator popping into your head.” – Joshua Bloom
“It should be very clear that the reason why we're doing this is because we more or less have to.” – Joshua Bloom
“I feel guilty for having stuff on my disk that you can't look at. It's the guilt. Maybe guilt is a great motivator.” – Joshua Bloom
HOST: The following lecture was recorded on October 4th, 2010. Our speaker tonight, Dr. Joshua Bloom, did his undergraduate work at Harvard, got his Ph.D. from Caltech in 2002. He was always interested in science as a child. He started off being very interested in computers and technology, not quite so much in astronomy. His interest in astronomy came a little bit later on — got more interested in math and physics, and then interested in astronomy, and then is now circling back a little bit to computers and technology. His specialty is a very strange and mysterious phenomenon called gamma-ray bursts, and he'll tell you a little bit about that, but his talk tonight is a bit wider in scope, looking at the amount of data that we can see when we look at the universe now through the surveys, which can show us changes that are taking place all throughout the universe, and over time, we can see things actually changing as we look. We can see supernovas appearing and disappearing, or gamma-ray bursts appearing and disappearing. We can see how things move. And so he's going to talk with us about that. And so, ladies and gentlemen, I'd like you to join me in welcoming Dr. Joshua Bloom.
BLOOM: Thank you very much for the introduction. You've had some time now to figure out what you're looking at. Who wants to tell me what we're looking at? Galileo's notebook — actually a composite of his notebook. That's right. What you're looking at is actually, in some sense, one of the earliest and most famous time-domain observations. This is a recording of those observations: Galileo looking at the planet Jupiter and finding for the first time some strange stars, which he depicted here. These are not actually stars. There's the depiction of Jupiter, just as a round circle, looking through a telescope. And what he wound up realizing, of course, is that these stars were not the traditional stars that we had come to know and love by then, but actually moons of this giant planet.
This had very important implications. And the reason why I wanted to show you this picture today is because it really captures some of the excitement we have about observations in time, but also highlights some of the very important roles that observations in time — or so-called time-domain astronomy — have had through the history of scientific discourse. There's a couple things to point out about this slide. One, of course, is that time domain is absolutely crucial in everything that we've done in the past and we're doing today. It also emphasizes the crucial role of humans in this process of discovery, of making observations, figuring out what it is that we're actually seeing, and then going on from there.
And there's another aspect to this, of course, which is that whenever astronomers, or scientists for that matter, get their hands on new tools — in this case a telescope, which was originally built to look over the horizon for coming threats, so it had a military application — turning not from the horizon but up to the stars, Galileo made transformative, revolutionary discoveries that changed our view of our place in the universe. These observations that you see before you here, taken over just a couple of months — 400 years ago, thereabouts, today — had an incredibly important role for ordinary people in understanding finally that the Earth was probably not at the center of the universe.
Many people had been thinking about this for a while. There were many observations at the time that were hinting away from this. But the idea that the universe — in particular, these wandering objects, these planets — were these perfect spheres, these perfect orbs, and everything around them were moving — basically there would be nothing around them, but they themselves would be moving around the Earth. There wasn't supposed to be anything around Jupiter. The fact that there was something around Jupiter, and there were four of these things, was a big problem. And so this geocentric model, or the Ptolemaic model, for the universe really started showing some of its major cracks in these observations that you see.
400 years later, the nature of astronomical discovery and inquiry is really undergoing a major phase transition, and that's the central focus of what I want to talk to you today. Not only do the objects of interest require larger and more sophisticated telescopes, the dynamic universe sometimes demands reactions on time scales faster than humans can collate information and respond. Couple that with massive data rates, and it becomes clear that the traditional roles of astronomers have to be changing, essentially by necessity. So today is really a discussion of some of the interesting science, some of the interesting things that are happening in time in the sky, but also our connections computationally, algorithmically, to how we're actually going to solve the next set of problems ahead of us.
What's really not changing throughout all of this is our sense of science and our path to doing science. What you see here is a very simplified picture showing you how we go from observation to scientific inference. This is a part of the scientific method, but it's not all of that. We make observations like Galileo. We find something, so essentially just recording what it is that we see. And then from that we make a great leap and say that this is something new, or it's something that we may have seen before, but not in that same part of the sky. And then we follow up, perhaps with other telescopes. Maybe we come back to that same place in the sky a little bit later on and see what's changed. And then we really start making our scientific inference.
Discovery in the past has really been a major bottleneck. Actually just finding these new things and saying this is new — not saying what it is, but just saying this is new — has been one of the biggest bottlenecks for the last 400 years, and that's something itself that is changing. One of the important things, this is: just finding something, maybe even what you would call discovering it by noting that there's something there, isn't really the same as our notion of having actually been the discoverer, or having been the person that made the grand scientific insight, that great leap.
And in fact, in those same observations that I showed you, Galileo happened to observe Uranus in his telescope, 20 times the power of what the human eye could see. And he recorded Uranus, but he recorded it as what he thought was a background star. Galileo is not credited with the discovery of Uranus. That came several hundred years later, because he didn't know what it is that he had. He recorded it, he saw it, but he didn't know what he had. So that great leap forward of understanding what it is that you have is one of the most important roles that humans have had in astronomy.
I want to talk to you today about this dynamic universe. And I think a lot of you already intuit just how dynamic the universe is, just how changing the sky is, not just on a daily basis. You can see, for instance, that Jupiter is rising at different times in the east these days, or that you might have comets that come and go over the course of your lifetime. But there's something else that you, of course, have intuited and know about — at least, that they happen — and that's supernovae. Supernovae are the explosions of very massive stars in some cases, but certainly related to the death of all stars at some level.
And this is one of the most famous supernovae that mankind has ever witnessed, and we've been fortunate to have had this happen in quote-unquote modern times, when we've had very sensitive instruments working at many different parts of the electromagnetic spectrum, and even different spectra looking at particles, to try to understand what caused this explosion. Somebody looked up in an observatory, Las Campanas Observatory in Chile, in February of 1987 and looked at one of the Magellanic Clouds, these satellites that you can only see from the Southern Hemisphere, unfortunately, of the Milky Way. And he saw — well, not this arrow, because that was actually a before picture. That would have been pretty remarkable. But he saw a supernova. He saw a new star, actually, and he very quickly, because he was an astronomer, realized that this was something quite special. And then a picture taken a couple days before, of what would eventually wind up blowing up, told us a whole lot about how the end states of that star actually came to be and how that wound up blowing up, forming a supernova — we had known about many different types of supernovae before that, but this was sort of our nearest neighbor in the modern times.
This is a very dramatic example of the dynamic universe. Supernovae in our own galaxy are quite rare. They happen, say, once every 100 years or so. We do see supernovae throughout the universe, and I'll talk more about that later on. But much of what happens in the universe, in this dynamic universe, is something much more subtle. And one of the things that we talk about when we think of the dynamic universe is change. And this change can be either changes in brightness or, as I've depicted here, changes in location. This is motion in space.
Hopefully you can see something special happening. Don't worry so much about the ripples; those are detector artifacts. What you see here is probably the most famous three-frame movie you've never seen before. I'll ask somebody to just yell out what they think this might actually be. I will tell you the answer, so don't worry. Close. I heard asteroid and Pluto. Probably sort of the geometric mean between those two: a Kuiper belt object. Yes. Good, so you guys are getting — it's something in the solar system, that's for sure. It's definitely related to the Kuiper belt. Here it is right here, moving. This is three sets of images taken over the course of about an hour. This object is moving so slow, it would take two and a half months or so to cross the face of the full moon. This is a very small, almost imperceptible change.
What's interesting about these images is they were taken with a very small telescope with actually a fairly modest camera. But somebody was very interested in understanding how the outer solar system was changing. So this is the so-called discovery movie of what's now called the dwarf planet Eris. It is now known to be larger than Pluto. So if you have a problem with Pluto being kicked out of the planethood, this is the movie to blame, and these are the people — Mike Brown and company — to blame for this. This was a major discovery, but you see it was just by noticing a small little change in a fairly random place in the sky.
Another very important discovery, which was more recent, showing actually another little small movement. This was some little object in 2004, and then this object actually moved, and this is sort of a composite image taken over two or three years with the Hubble Space Telescope. There's where it was in 2004; there's where it was in 2006. And way off to the top here is an incredibly bright nearby star called Fomalhaut, which has been masked out here, and you see a little blow-up of this region here. This is almost nothing. This is a dinky little piece of light. But what you see here is the motion along an orbit that would be consistent with a planet around its parent star. This is the discovery image, the first images ever taken by mankind of a planet outside of our solar system. And again, this discovery happened — actually, the data were taken not because people were looking for planets, because they had other things in mind. But someone observed this and realized that this actually was probably a planet. So fantastic and revolutionary discoveries happen because of our ability to make measurements of motion on the sky.
As I also noted, one of the things we can do is take pictures of random stars, or stars of interest to us, and measure as a function of time. Here we are, time progressing in days. This is — don't worry about these, the actual numbers here. This is five days, so this thing over here is of order something like a month or so. And it doesn't matter so much, the scale here. Each one of these little points is a measurement of the brightness of these — something like six stars that you see here. What's remarkable about this — because it just looks like these stars are burbling along. And by the way, our sun also burbles along too. It's not a fixed brightness in time.
What's remarkable about this is, first of all, that we can even see this burbling, because this burbling is happening on levels of parts per million. So this would be the equivalent of, say, flying into Las Vegas and noting, since the last time you were there, that somebody had their light off in one of the hotels. This is incredibly small changes of light that we're now able to make with actually a fairly traditional telescope, but one that actually has been launched into space, so we don't have to worry about a lot of the atmospheric effects. This is transformative stuff, even though all we're looking at is the general flickering and burbling of essentially ordinary and random stars. Many stars actually do do much more than this.
But one of the things that's absolutely so remarkable for us is that when we think about change and we think about ordinary stars — we look up in the sky, we look at any of these stars, and to our eyes, of course, they twinkle, but we know that's an atmospheric effect. But to our eyes, over the course of our lifetime, these stars don't change. But every single star changes its brightness, and every single star moves. Thinking about the universe, and thinking about our place in the universe, as something that's static and unchanging is the wrong way to think about it. Instead of taking pictures of the sky, what we're really doing, and what we're really interested in, is taking movies of the sky. This is celestial cinematography. Thanks.
Let me talk about, just for a minute, what it is that we learn from changes. What might be obvious to many of you is that, unlike when we look at, say, Jupiter or Saturn with a telescope, where we can actually see structure on top of this object, on top of this planet, for the most part the objects that we look at more or less look like they're just a single point of light, and what we're measuring is the total light coming from them. That's how we measure these changes in these light curves. We can't usually, what's called, resolve most of the objects of interest to us. So when we look at a very distant star, we can't see that there's a sunspot over here and there's another sunspot over here and that it's rotating around. All we really have is our measurement of this brightness as a function of time.
And what's nice about this is there's a very intimate relationship between these brightness changes and the total sizes of the objects, or places on those objects, that are involved in those changes. This is the importance of the time axis when we're collecting data, and this speaks very much to the physical notions of the total sizes involved. So what we talk about here is a time scale. This is sort of the overall time for something, say, to go up and down by a factor of two in brightness. And we might denote that with so-called delta T. And this time scale is related to the size of this emitter.
So here is some sort of generic object, which is going to have some sort of hot spot on its surface. Maybe this is a disturbance, some sort of perturbation, or maybe even a cloud or rainstorm on the surface here. Or maybe this is some grand explosion that starts off in one little place on this surface. And this disturbance is going to wind up moving outward over this surface, propagating outward over the surface. And most of these disturbances will actually travel fairly slowly, but we know that there's an upper limit. The fastest that these disturbances can move is something like the speed of light. So if I associate this distance with a size, which I'll call S, one of the hard and fast rules that astronomers take away from time series, by looking at times of change, is that we have a size S, and this is less than the speed of light, which is denoted by this constant c, times delta T.
So that means that if I see a disturbance from an object and it has a time scale of change of something like 10 minutes, I know that the size of the region that's responsible for that change has got to be something less than the Earth–Sun distance. That's what 10 minutes translates into, roughly, when you do the conversion using the speed of light. This allows us — without being able to resolve and actually image and take pictures of these individual stars — it allows us to infer a great deal about what the physical origin is of all the different changes that we're seeing.
And we've learned over hundreds of years of astronomy that there are many different types of variability. There's pulsations: stars like the sun — in fact, all stars — will pulsate at some level, breathing in and out. They will rotate; stars like the sun spin, of course. And many stars happen to be in an interesting dance, a binary system with another star, sometimes much larger, sometimes smaller. But if we happen to be oriented just right to the orbit of that binary system, we may actually see the light of one star wink out a little bit — or, we say, eclipse — and then later on we might see the other star move in front of the other one, and we wind up seeing another eclipse. So binary systems can block the light temporarily. And of course, to see an eclipsing system, you have to take images as a function of time, and you have to notice that there are these brightness changes.
There's also episodic changes, which could be related to the mass, or the amount of material that's falling onto an object, and maybe that object is spitting some of that stuff out: unsteady winds or mass flow. And some of the most dramatic ones, as we've talked about, are explosive events like supernovae that can destroy the original object. There are hundreds of classes of variable sources and events. From supernovae to stars like our own, we can put them into different buckets and try to understand the interesting physics of why there's different classes of this variability. More than just putting things into buckets, though, there are obviously lots of different ways in which we get excited about the astronomy of the time domain.
And I want to just highlight my view of the three different realms of that — the intellectual thrusts in transients. One is the introspection. It's the study and the origin of the relevant physics giving rise to the class of the events themselves. Another one is exploitation. It's saying, I really don't care what those objects are. I don't really even care what the physics is that gives rise to the changes that I see, but I know that I can make use of those changes to help me learn something about, say, the universe or very basic physics. So using them as in situ laboratories or lighthouses or beacons.
To give you a sense of what we might say as an in situ laboratory, there are these objects called neutron stars, which are very dense cores, remnants of previous stars. They're spinning very fast, they're magnetized, and they sometimes can put out a great amount of light, but the objects are spinning, and so we think of these things as lighthouses. This is what we call pulsars, and we can observe these at radio wavelengths. The physics of what gives rise to this pulsed radio emission is actually very complicated. Many people have come and gone over the course of the last 40 years trying to understand the physics of pulsars, and we really don't have our head around it completely as a field.
But what we know is that because these stars are rotating so fast and because they are so massive, they really don't change the rate of the rotation all that much. So we can use these pulsars as clocks. It turns out if you have very, very precise clocks in the presence of a large amount of mass, you can actually test some interesting tenets of general relativity. And so pulsars and the observations of pulsars have been some of the best tests that we have of Einstein's general relativity. That's an example of an in situ laboratory, where the science or the physics of what we're interested in is really disconnected in many ways from the physics of the objects themselves.
Another one might be a lighthouse or beacon. Say I find something — let's say a gamma-ray burst, a very massive explosion, even more energetic than a supernova. I can see these objects to the edge of the observable universe, and even though I might not understand in great detail the origin — the physical origin — of what gave rise to all the things that I see, I happen to know that that place in the sky is a very interesting place, and that may actually be the location of some of the first stars and galaxies in the universe. So I can say, it doesn't really matter what these objects are. I just know that they're bright, and I can see them very far away, and I'll make use of that idea in and of itself to exploit these variable objects.
And last — and this is something that gets many of us out of bed every day, when we get despondent and we don't know what we're going to do with our lives — is exploration. It's this idea that because we're looking at the universe in a new way, we may actually find something either that somebody's thought about before — hypothesized or theorized about — but we may also find something that no one's thought of before. This sort of Wild West mentality of going off and looking at data, not really knowing what you're expecting to see, searching in parameter space, searching in ways that people haven't searched before, is very exciting, and in the time domain it's one of the last frontiers for doing this in astronomy.
Let me give you a couple of examples from my own work of these various different ways in which we think about doing work within the time domain and with transients. Here's an exploratory set of work that I undertook using a survey called the Sloan Digital Sky Survey. It was a repeated scan of the same place on the sky over the course of four years, taken in the fall months about once every day or two. And it was taken over a region of the sky — 290 square degrees, something approximate, 1,500 full moons or thereabouts. So not a very large part of the sky, but it was taken with a very sensitive instrument and it was done repeatedly, so this is where the time domain comes in. And it allowed us to look for objects that were changing.
And because all this data was more or less put into a database that became public, we could just look at that data, even though we weren't directly involved in the survey itself. And even though the data itself were taken for the express purpose of finding supernovae, we thought we might be able to look for eclipsing systems by looking for the winking off of some stars. And we focused on very, very low-mass stars called M dwarfs. And we looked at something like 20,000 light curves. So this was 20,000 anonymous, low-mass, red stars.
And what you see here as a function of time and as a function of brightness in these four different colors — more or less a steady source, right? It's not changing in any of these different colors, except there are two times when there were these little dips in the brightness. And those dips happen more or less simultaneously in all these different colors. This is an eclipsing system. We didn't know what we had, but we knew that if we could throw some resources at this discovery, we might be able to infer what's going on.
And this is where the introspection came in. All of a sudden, this very large field on the sky — of that entire place on the sky, this one place on the sky became of great interest to us. And we're able to look at those objects in great detail. We're able to find — if we place the location of the two objects, indeed it was an eclipsing system, in units of mass of the sun and units of radius of the sun, you see that we're much less than the mass of the sun, and we're much less than the radius of the sun. We're way, way down on this curve here. And what you see in all these color things here are all the theoretical curves. And these are some other very famous eclipsing systems. We were able to find some of the lowest-mass, smallest stars just by going through a public data set and trying to figure out which objects were interesting. And we were able to place interesting constraints on these theoretical curves that have been around for decades. And we have something like 30 or 40 more of these objects that we're interested in and we're studying. So just with a first-year graduate student, we've been able to open up a fairly interesting part of parameter space using a public data set.
So this is the example path towards science here. We have the observing and the finding being done by this very large telescope — actually a 2.5-meter, not large in the context of other things I'll talk about. And then discovery, just more or less writing specialized computer codes to figure out which systems were going to be of interest to us to look for eclipses. And then we went to various telescopes by ourselves, and we went on mountaintops and took data to try to figure out what was going on. And with very sensitive facilities, we were able to see a very beautiful picture of this eclipse. This is a dip in the light here. This change is only about 20%. And underlying here, you might not be able to see it, but this is telling us something about the masses of these various systems. This is the example path, not so different from the way things have been done for years.
And then exploitation — I already mentioned one of those in the context of gamma-ray bursts and also, I guess, with pulsars. This is one of the most famous exploitive ways of using transients or variable stars. It's a discovery made by Henrietta Swan Leavitt in 1908, where she realized that there were certain types of stars called RR Lyrae — sorry, Cepheids — and those Cepheid stars had a very regular pattern to them where they repeated. These are what are called periodic stars. And what her great insight was, was that depending upon the period — this is now shown here in days — these stars were intrinsically of different brightnesses. In fact, there was a one-to-one correlation between the period and these brightnesses, which is very important because it means if you can make a measurement of the period of one of these stars, you can actually infer its intrinsic brightness, and because you can infer its intrinsic brightness, you can figure out the distances to these objects.
Finding the distance in astronomy is one of the hardest things that we can do. When you make a measurement of light as it hits your detector or your photographic plate, what have you, there's nothing that is encoded in that light itself that tells you how far that light traveled. It could be coming from the edge of the observable universe or somewhere within our solar system. So being able to make measurements of distance is one of these great things in astronomy, and this was one of the great discoveries that allowed us to finally start mapping out the total size scale of the Milky Way and then the distances to the nearest galaxies.
Of course, some of the greatest distance indicators today are these events called supernovae, and supernovae, because they're just so incredibly bright, can be seen also to very, very large distances. This is a little artist depiction of a supernova going off in an anonymous galaxy, which is about 10 billion light-years away, and then we zoom into something which isn't an artist depiction — an actual picture. We zoom out farther and farther out. This movie here gives you a sense of this — while the supernova went off in this galaxy, and we got very excited about that position on the sky, there are a lot of those galaxies in this whole picture that you see in front of you.
And so even some of the most rare events — supernovae are incredibly rare. Only 1% of stars will ever blow up as any form of supernova. And as I said, there's only about one supernova in our galaxy every 50 to 100 years. These are very, very rare events. I just told you that every star is variable in its own way, but very few of them will die as fantastically so as supernovae. Yet because the universe is so vast, there's about one supernova every second in the universe that goes off. Why is it then that we only detect about a thousand supernova a year as mankind? It's because we're just not looking in all places at all times to all these different depths that we want to see to be able to see all supernovae. So the universe is teeming — even the most rare objects, because the universe is so unbelievably vast, these things are out there to be found.
This, in the parlance of a former Bush administration official who shall go unnamed, might be called the known unknowns. We know that these things are out there, like supernovae. But what about the unknown unknowns? What about the things that we haven't even thought of yet that might be out there and are still waiting to be discovered? Theorists have started wrapping their heads around this, and they've started making diagrams like you see here. It's not important that you know all the different terms up here. Just to show you that there's a gallery of things in other galaxies that can blow up and produce detectable light for us, some of which we know, like Type Ia supernovae, for those that follow the supernova game, or Type IIP supernovae, so-called, which are the most common form of supernovae in the universe. And then there's the mythical pair-production supernovae, which is an explosion by a very different physical process.
And then there's these other things like a neutron star–neutron star merger, which are quite important for this upcoming decade as we try to find for the first time the actual measurements of gravitational waves. Neutron stars will wind up smashing into each other. These binaries wind up colliding, and it could produce a very small amount of detectable light just for a short period of time. In some sense this black line here used to be called a mini supernova, because it's a lot like one of these other supernovae — much fainter, and it lasts a lot less in time. So one of the real excitements scientifically is us being able to make these discoveries in the time domain of objects we know about, and we know we can exploit, and we're really interested in their physics, and objects we don't yet know about.
So I want to now turn our focus not so much to the objects themselves and what we can do with them, but the process of how we observe the universe. How is it that we go about making these discoveries and doing science within this changing universe? I like this picture a lot from Vermeer — famous painting called The Astronomer — in part because it shows you just how easy it used to be to be an astronomer. You could do your work during the daytime. You can look at charts of the heavens made by other people, and you could have these grand insights into the universe just by sitting on your chair and not actually doing any real observations. I guess this is the way that the gentleman astronomy used to work way back when.
A traditional yet fairly modern view of how we do astronomy these days is this depiction of Edwin Hubble. I guess he's also not recognizing that it's pretty much during the daytime, or at least somebody has the lights on in the dome, and he probably can't see anything out of here, and all the interesting equipment is down here, and you probably shouldn't be smoking. There are a lot of things wrong with this picture. This is one of the most famous pictures of Edwin Hubble. But in Hubble's day, some of the grandest discoveries — in fact, all the grandest discoveries at optical wavelengths, the visible light that our eyes can see — were acquired and obtained with photographic plates. Basically, this is just a glorified camera here, and the film that was used was a glorified version of 35-millimeter film.
One of the great advances in our ability to do our work as astronomers came with the discovery and the usage in astronomy of these so-called charge-coupled devices, or CCDs. It's this movement from photographic plates to these CCD imagers. This is actually two different cameras separated by something like 30 years in time, and this is an astronomer who's putting in a photographic plate at the primary focus of one of these telescopes, and this is the camera that actually replaced this one here on the same telescope. It's called the Large Format Camera. It's at Mount Palomar, and it covers about a full square degree on the sky. So actually, a lot of pixels here.
These CCDs — these detectors — have very high quantum efficiency. They collect light at something like 50 times the previous rate of this photographic plate here, which means an image that used to take something like two or three hours to acquire now takes of order a few minutes. That is a dramatic revolution in our ability to acquire images of the sky. This is a great revolution, and in part because of the revolution that this caused in astronomy, the Nobel Prize was awarded to Boyle and Smith last year for the invention of an imaging semiconductor circuit, the CCD sensor.
Another thing that's changed a lot — really a mini-revolution in the mid-1990s, excuse me — was this evolution from humans being part of the actual data acquisition to robotic telescopes being part of that data acquisition. Here I am with a graduate student looking at data and moving a telescope around more or less by hand, although there are mechanics to do all that stuff. But just being up on top of the mountain taking data is our traditional view of what astronomers do. But it's this motion from human operation to robotic telescopes.
This is a telescope that's operated by Alex Filippenko and his collaboration — the most successful discovery engine of supernovae ever made. It's been operating for over a decade at Mount Hamilton. A small telescope — I think 70 centimeters in diameter — by modern standards, but dedicated to this one scientific endeavor. And so this robotization — taking people out of the loop — meant that this telescope got to operate very efficiently without the possibility of human error on a daily basis, and was able to obtain data at a much higher efficiency rate than could be done if you had humans part of that loop. Here's me, looking somewhat smug, in front of my robotic telescope, something that I've worked on for a number of years. And we've changed not just the way in which the hardware works to actually just take the data and send it down the mountain, but we're trying to create these more intelligent collection agents, and this is something that I've had very near and dear to my heart. One is actually scheduling what we decide to look at as a function of time, autonomously, where we have an armchair astronomer — going back to the Vermeer picture — just dialing in on a website what they're interested in, and letting some big computer code crunch and decide what's interesting, optimize over all the different projects. We have detailed weather sensing on our facility. I should say that this is something that's out on Mount Hopkins in Arizona, just south of Tucson. So this telescope can know when weather's coming that's not going to be good for it to be open. It'll close itself down, and know when weather gets better and improves, and so it can open itself up and start taking data again.
And for the purposes of the things that I've worked on for a number of years, these objects called gamma-ray bursts, these telescopes are able to quickly react to new events that happen in the sky. What's happened is there's a network of satellites that are capable of sending data down as they discover new events in the sky, and those different triggers are able to actually get our various telescopes moving and taking data in various places on the sky of interest with more or less no downtime. Some of our best examples of a gamma-ray burst, which occurred on essentially the other side of the observable universe, was detected by a satellite called Swift. That satellite more or less sent a text message to our telescope, and within 50 seconds or so we were taking data on that place on the sky because we knew where to look.
So we have a bit of a cheeky website where we call ourselves the robotic transient astronomy for the 21st century. You can see all our various discoveries here. But you notice on the right-hand side is that this telescope actually twitters itself, and you can follow the Twitter stream of this telescope, PAIRITEL, and you can see what it's looking at. So we are trying to keep up with the times. This idea of robotic telescopes and the robotization of telescopes has been around for a while, but it's this idea of these intelligent agents — these telescopes were able to more or less do the thinking by themselves without anyone else in the loop deciding where to look at — has gotten obviously the astronomy interest very high, and it's something that people are starting to look towards as one of the major ways in which we can be efficiently observing this changing sky.
So one of the challenges upcoming is how do we get smart telescopes to talk to each other and federate themselves to do interesting science while the humans that own them are actually asleep. I'm sorry for those of you that are having images of the Terminator popping into your head. That's not my intention. But it is actually somewhat of a reality. One of the problems that we've had in the past is that the way in which astronomers talk about the changing sky, talk about a new event, is they announce these events. It used to be over essentially teletype and faxes, telegrams. It wasn't so long ago that people would be showing up from the Pony Express telling somebody about an interesting object that somebody found a few days ago or a few weeks ago. It's not that much different now. Now we have email.
But of course, when somebody says, “I found something here,” they talk about all the different things they got excited about. They show where it is in the sky. They might mention a couple things of what they think it is. This is, in this case, the announcement of a discovery of a young supernova of Type Ic. This gets broadcast out to all the people that want to hear about this. But one of the big problems with it, and this way of announcing discovery, is that: what do you do with this? If all you have are machines, you'd have to have a machine go through and parse this and figure out what's the interesting part in the sky to look at. Am I even interested in this object? Do I waste my telescope time on this, or is this the most important thing ever? This human-written messaging scheme is something which is very obviously outdated in the context of a universe which is changing a lot, and we're gonna be getting a lot more information about that changing universe.
So another thing that I've been working on for a number of years is to get a standard way of reporting messages of transients in the sky. And it's a messaging scheme which has now been blessed by the IEEE of astronomy, called VOEvent. And all it is — this is certainly beyond the scope of what you'll be tested on at the end of today's lecture — is more or less just a who, what, where, and when, and how, but it's written in a language that all machines can actually understand. So because we've been able to create this messaging scheme, we've been able to create a framework for doing follow-up and science with these robotic telescopes.
And again, you don't have to understand or know what all these different phrases here and different acronyms mean, but the point to show you is that throughout the world now, people are starting to go on to this messaging scheme, and what you wind up seeing is a topology which looks a lot like the internet itself, where you have these central nodes of information and various telescopes which are feeding into that, and various telescopes which are on the receiving end of that information. And this is a growing, kind of living network that is able to respond to transients in real time and take data without anyone jumping in and telling it what to do. So we've been very involved in the community in getting these robotic telescopes not only networked and roboticized themselves, but actually talking to each other and doing interesting work. This looks like they're doing bad stuff, but most of the telescopes we try to work on are doing good things.
That's on the follow-up. And if you remember my diagram of this pathway to science, and I said discovery was a big problem — what do we even look at, what do we throw our telescopes at when we have this network that's capable of responding? Well, that's where the big story is coming. That's where the biggest changes are going to be happening in the next couple of years. One of the major projects that I'm involved in, thankfully, now is called the Palomar Transient Factory. It's operated at Palomar Mountain, and the goal here is to characterize the transient and variable sky. And this is using a camera that has something like 100 million pixels on it, observing a very wide field. I showed you that one that was covering something like the full moon; we're covering about 30 or 40 full moons at any one shot. And we're taking images just for one minute, and then we're moving on to another place in the sky. We're able to obtain thousands of square degrees of images of the sky every night.
And our primary goal is to just observe how things are changing. It's getting at some of the bread-and-butter science, like looking at these variable stars, but also looking for new things. And this is going incredibly well. We have a bunch of different candidates here. You see all of these different objects highlighted with these purple crosshairs. We've got something like 15,000 new objects in the sky that we've identified for the first time, just using what turns out to be a fairly small facility.
But the real reason why we're getting into this idea of discovery on a large scale is because of a survey called the Large Synoptic Survey Telescope. This is an 8.4-meter telescope whose mirror, by the way, has already been cast. It's a well over half a billion dollar facility that some of your taxpayer dollars will hopefully be paying for. And the amazing thing about it is just how gargantuan the data rates are, and its grasp on the sky is just mind-blowing today. The Large Synoptic Survey Telescope will collect something like a gigabyte every two seconds, and that translates into something like the discovery of a million supernovae a year, something like 100,000 eclipsing systems like the one I showed you already — remember I said we only have about 20 or 30 of them of interest; that will far surpass that — something like 10 million asteroids every year. It's absolutely amazing to think about not only the science of what will be enabled by these discoveries, but just the sheer data rate. And the real big challenge that we have is how we come to understand, discover, and then actually make scientific inferences on these light curves that will get close to something like a billion light curves every three days of interest.
So to come back to this a bit, showing you this pathway to science, where I said that the discovery process for the longest time has really been part of this bottleneck. LSST is this 800-pound gorilla — or whatever, 800-pound or 800-ton telescope — in the room that's really going to change all this. Because discovery now — the actual observation and actually putting down on the paper that this is a new object — that becomes the easy part. It's this follow-up part. You all of a sudden have a new bottleneck where there's not going to be enough glass on telescopes in the entire world to be able to follow up all the stuff that's interesting. So here, trying to come to understand what the really interesting stuff is becomes a very new challenge. And because the data rates are so high, there's no way that we could have ourselves, let alone our undergraduates, looking at all of this data on a daily basis. There's just too much of this stuff.
How do we do discovery, follow-up, and inference when the data rates and the requisite time scales preclude human involvement? It's not just the data rates. It's not just the difficulty of getting human eyes looking at this stuff. It's the fact that I can't wait until two years from now to actually go back and realize I had a very important discovery of one of those mini-supernovae that actually required a lot more observation by bigger telescopes, and I just didn't have the manpower at the time to actually know that I had that object there. It's our own Galilean Uranus problem, where we're going to have this stuff in our data and we won't even know that it's important to look at, because we don't have any way to actually get at that data and actually go through the needle in the haystack. This problem — or this challenge, I guess you're supposed to call it — is not one that is owned by astronomy alone, but is something that, here, the Economist article I think very beautifully depicts, is now being faced with and dealt with by a number of other different not only scientific endeavors, but social endeavors as well, let alone the business side of the data deluge.
So how we handle this data deluge is something that I've started getting very interested in. And we've been fortunate enough to build up a collaboration with a number of people, not only at Berkeley, but now at other institutions and other groups you may have heard of, allowing us to bring in some of these tools. Because, you can imagine, we're looking at so much data, we have to deal with computer science issues, but we're looking at light curves. We're looking at how things change, and characterizing that change and classifying that change gets into very interesting realms of machine learning and statistics. So we've gotten some computer time donated by Google and Yahoo, and the National Science Foundation has started getting behind this idea of the need to build up a framework for being able to sift through this data, this needle in the haystack. So our goal here — and I won't go through all of these here — is more or less just what you see at the top: autonomous creation of new knowledge that itself spurs further follow-up and inquiry.
Going through these images, this is an actual image from the Palomar Transient Factory. Here's a picture to scale of the full moon. There are something like 100,000 stars here, and a comparable number of galaxies in this image. You won't be able to see them all because they're much fainter than these very brightest stars. But how do we go through and just find new things, right? How do we go through and find the objects that move or find the objects that weren't there before? The way in which we do this is we have previous pictures of the sky, and we more or less subtract that picture off from our current picture. And this so-called subtraction image allows us to see things that have changed and moved, right? Because if I had a picture of the sky and nothing changed in the sky, and then I came back a year later and I took the same picture of the same part of the sky, if I subtracted, all I would get would basically be noise. It would look like the fuzz on your TV. But instead, we see a lot of things.
The problem is that even though we're using state-of-the-art subtraction techniques in this survey, there is something like 1.5 million candidate new objects a night that come out of our data stream just today. This isn't talking about LSST. This is happening night after night now. And probably only about 1,000 of those are truly bona fide sources, and most of them turn out to be asteroids, and probably only 10 new variable transients. This is a huge needle-in-the-haystack problem, 10 out of 1.5 million, and it's obviously very clear that we can't get enough people looking at 1.5 million objects.
So one of the things I've been working on is figuring out a way to get human judgment to be part of the actual discovery process of these different objects. We created a website where we have experts looking at these subtraction images. Here's the original image, or the so-called reference image, the new image, the subtraction. This actually looks pretty bad. Don't worry if you don't know that right away. This one down here looks pretty good. So what you have is we just ask people: is this good or bad, bogus or astrophysical? And they just do these slide bars on a few hundred of them. And we use that information from about 30 or 40 experts, quote-unquote, to train a machine to look at these images and come up with their own inferences about what we're looking at — whether it's good or bad.
And so what's interesting is, out of the first month of data or so, we had something like 20 million candidates; most of the stuff that we were looking at was quite poor. This is on some scale of being really bad and really good. And we had something like 50,000 real objects out of all this up here. So this has allowed us to drill down into this imaging data in a new way, basically taking people out of the discovery loop and finding transients. And here's a movie of the location of something like 10,000 transients and variable stars on the sky. This would be us sitting in the middle of this looking out. That red dot there is the Andromeda galaxy, that we found a lot of transients there. And this is, in some sense, just the precursor to what the Large Synoptic Survey Telescope is going to do.
But again, discovery is not the same as doing that scientific inference and actually understanding what it is that you're looking at. I hope that's one thing that should be clear from now. Just because I see something go up and down doesn't tell me that it's a supernova or an RR Lyrae star or a Cepheid or something else. And you'd think that by looking at this large class of all the interesting variable stars — this is a depiction of that taxonomy. Doesn't matter if you can't see all that. We have some great examples of different types of variable stars. We have Mira variables, supernovae, something called Beta Lyrae, and then Ursa Major. These are very common variable stars. You see they vary on different timescales. This is a timescale of just a few hours. This is a timescale of years. So things are changing on all these different timescales at different brightness levels. And you'd think that we could use all these to help us get a sense of a new object that we look at and what class it actually belongs to.
But one of the really interesting things when we bring this problem to the machine learning people, and those that work with time series data — so we're dealing with very large amounts of data coming at us where we have to know what it is that we're looking at. This is obviously something of interest in the finance world. But the difference between us and the finance world, in large part, is that our data is pretty bad. Our data is pretty noisy. So this is an example of a so-called microlensing event, where you have a star which is more or less not changing. And then all of a sudden it has this little interesting blip here. This red line here is the model of what we think we should be seeing if we had perfect data. But what you see is that there are all these different data points here. And these little bars up here denote our uncertainty of exactly the brightness at that one time. There's a lot of noise on here, and there's actually gaps in the data. So even if we had a perfect model of what it is we're seeing, we've got imperfect data.
So this is a big problem in the machine learning world, and they look at this data and they say, “Wow, your data isn't regularly sampled, and it's noisy.” They get very excited about this. The other problem is we have spurious data, right? We've got actual data points which are wrong. They made it into this light curve, but they shouldn't have made it in. We also have — the interesting part of the event may not have happened yet. The interesting thing that is a telltale signature of what class that object belongs to may not have even happened yet. So we have to have models, forward-looking models, of what's going to be happening in the future. This is actually quite interesting.
I want to skip over a couple slides here, just to show you that even though I've said that humans really can't be part of this real-time loop, we're still using them in interesting ways. This is the so-called Galaxy Zoo project. It's an interesting website if you haven't visited yet. It allows you to actually look at Palomar Transient Factory data to look for supernovae, where you are presented with the same picture that I presented the experts. But this is happening where every night we present about 100 of these different subtractions. And we ask very basic questions, and based on what people answer, we wind up going after these interesting candidates to see if they are indeed actually supernovae. And the Galaxy Zoo, looking for supernovae, have been very, very efficient in finding supernovae that our machines aren't great at doing at this point. This is one of the things we're heading towards.
But one of the things that's important — this is what's called crowdsourcing. You've probably heard of this in the context of other fields: the idea of getting lots of people involved, looking at lots of data, and coming up with the aggregate answer. So more or less you're not using machines anymore. Here the machines are the vector to get more eyes on this, but they're not involved in the decision process itself. The real problem that I have with this is in the LSST era, where we're going to be looking at petabytes of data — having people in the loop really becomes almost untenable, right? It's not just the understanding that I can't fit enough undergraduates to fit in the room. There may not even be enough people in the world who are interested in looking at LSST data to get to all the interesting objects that we really want to sift through.
So let me just show you a couple more slides of our approach now, because this is all just the beginning of how we use machines in this discovery process. There are really two different kinds of data that we wind up getting. There's the light curve data itself, what we call the time series, and then there's the context. It's where this event happened on the sky. And context is very important when you don't have a lot of time series data.
In the following sense: if one of you came up to me afterwards — which I encourage you to do — and you said to me, “Josh, I found an object last night. It was an interesting event,” I wouldn't necessarily ask you, “Well, was it getting brighter or fainter?” I would say, “Well, was it in the ecliptic plane, where all the asteroids hang out?” And you'd say no, and I'd say, “Well, was it in the galactic plane — that is, in the plane of the galaxy?” And you might say no. And I'd say, “Well, was it near a galaxy?” And you might say yes. And I'd say, “Well, was it in the middle of that galaxy or on the outskirts of that galaxy?” And you'd say, “Outskirts of the galaxy.” And I'd say, “Is it a red galaxy or a blue one?” And you'd say, “Red.” And I'd say, “Well, it's a Type Ia supernova.” And I'd be right 99% of the time. Here, I haven't told you anything about what happened. I just told you where it happened and what it's near.
So what's interesting is that in the limit of not a lot of data, context is very important in understanding the classification of an object. Where it happens is very important. And then over time, as you get more data, the detailed observations of the changes of brightness and even the changes of colors become interesting. But again, I've shown you these light curves that are noisy and they're irregularly sampled, and how you just try to fit all this data that's obtained in very different ways into a single box is not at all obvious. One of the things we've been trying to do is take this heterogeneous data and do what's called, in the machine learning world, feature extraction, that takes these disparate observations of data and tries to corral them into real number lines — actually, real numbers that populate a real number line in multi-dimensions.
And this feature extraction, as we've learned, is sort of a black art. But it allows us to take all this heterogeneous data and corral it into something that a machine can look at and see essentially all the same, and make inferences about what it is that we're seeing based on just that data. So you don't have to know all of these different ways in which we do this, but we have lots of different metrics that allow us to take these various data sets and put them all in a common language. This, by the way, is how, in part, Google does search. They take an entire web page and they essentially transform a very large web page into a multidimensional axis, so that when you do search, all you're really doing is looking for something that's near, in this multidimensional space, the search that you've just initiated.
And so we know this has been working for a number of years, being able to classify objects. Now, this is not the discovery part, right? I want to be clear: this is not how we say that that's a new, interesting place in the sky. It's saying, given that that's a new, interesting place in the sky, what is that object? There are many different variable stars. Again, you'd have to know all the acronyms. But if you look at the period of these variable stars, for instance, if they are indeed periodic, it's very clear that we get separation in that space.
Let me just skip ahead a couple of slides to show you how we use actual real data on real types of objects, real classes, to come up with inferences about what new objects actually are. Say I've got a light curve here, which is a time series, showing me in this case a supernova. I got its context, so I know where it went off in the sky. And I'll say that this is an instance of a class of blue triangle. I can derive a lot of numbers — doesn't matter what all these things are — but I can derive a lot of numbers off of this data here and this data here. And I can do this for a whole set of different supernovae, or I guess in this case a class blue triangle. And I can actually look at the distribution of all these different numbers in this space, so it allows me basically to look at histograms.
And then when I have — I should have not had this as a blue triangle — some other different type of object, I can look in these various different parameters at the differences between these objects in this very abstract space now. And you can see that in some cases, for this parameter, whatever it is, if I had a new instance and it showed up here, there's no way that it belongs to this class here; it very likely belongs to that class there. So we're really now in a very fun stage in this project, where we're just taking old data sets and old time series surveys, bringing it all into a common format, and just playing around with the data and seeing what machine algorithms actually give us the best classification.
I want to end with something that's, I think, pretty interesting, where now the machines, I think, are going to actually help us with doing science and actually help us with understanding the physics of these various mechanisms. This is a depiction of part of this very complicated classification tree that binds all these different types of variable stars together. But it's a mess. This classification of variable stars has been around for hundreds of years, and as you can imagine, there's a mix of phenomenology — that is, what it is we see — and our understanding of the physics of what gave rise to what it is that we see.
And so we have this very strange set of classes. We have things called pulsational variables, and they have a bunch of different subclasses, and eclipsing variables — that sort of makes sense. But just because I see a dip in a light curve doesn't mean it's eclipsing. It means physically I infer that that object is eclipsing. And likewise, because X-ray astronomy came later, there's an entire major class of variable stars called X-ray. That's actually completely ridiculous, because what gives rise to X-ray emission is a very heterogeneous set of physical conditions. And so we have this very strange taxonomy that's certainly a hybrid of the phenomenology and the physics. And it's actually incomplete and inaccurate. We know that.
So it's turning this whole problem on its head. We asked this, I think, interesting question: can the learning process — trying to learn how to classify objects that we think we know — can that actually teach us about the intimate connections between variable objects that we thought belong to different classes but perhaps are actually physically close to each other? Can we actually use our machine learning process to teach us about the connections — the physical connections — of these different variable objects?
So this is a very complicated slide, but one that I wanted to show you. In the whole parlance of how people do classification, you've got a whole bunch of different classes of variable objects — I don't know how many we have here, something more than 20. And we have a whole library of instances of these various types of classes, and we then can throw out something like 20% of them, build a machine learning algorithm that's good at classifying, and then say how well we did on that other 20%. And if we did great, then we'd always get the answer right, and there'd be a lot of power here — there'd be a lot of red along this diagonal, because every Mira variable would be classified as a Mira variable. In this case here — it might be hard to see — we got only 93% of those right.
But you notice that there's some power here that's off the diagonal. There's some down here. There's some down here. There's some down here. The way in which we've arranged these various categories — you can see it faintly here in different colors — is that the physics that gives rise to these different classes, these subclasses, is all about the same depending on how close they are on the right-hand side. And so the fact that our machine learning algorithm was confused by whether something is a Delta Scuti or a Beta Cephei or something like that is actually really interesting, because it means that even though we didn't imbue any physical understanding of the nature of these variables in our algorithms, we more or less discovered that they're actually fairly closely connected to each other.
So let me summarize the talk and saying that we've come a very long way in astronomy. This automation of data acquisition and resource federation is happening now. But just because we have this available network of robotic telescopes, ready and willing to do follow-up, doesn't mean we know how to use those efficiently. Knowing what to follow up in the era of the Large Synoptic Survey Telescope is absolutely key, and we think that autonomous classification is an important part of that.
And it should be very clear that the reason why we're doing this is because we more or less have to. There are some interesting avenues that I pointed to at the end here, that there may actually be new physical insights that come from this, but we simply won't be able to keep up with the data. And this is happening more or less now — in the next five years, ten years from now, as LSST really gets going. The role of astronomers is really going to change. We're abstracting the role of astronomer much further away from just being at the telescope and taking data — certainly not being part of looking at the photographic plates. But not only are we taking the astronomer away, unfortunately, from looking at the data, but we're also having to remove the astronomer from that grand goal of being able to do discovery and scientific inference. So it truly is a revolution that's coming.
And one of the things that's attracted me so much to this entire field is obviously the science of what we're getting at, of the physical objects of interest, but it's a truly rich interface between the machine learning world, the computer science world, statistics, and astronomy. And we think that some of the algorithms and frameworks that we're developing here will have grand applicability to other data-driven fields.
So let me end with a depiction — I guess it's an etching — showing a bunch of people pointing up in the sky and looking at this thing right here. This is a depiction of Tycho's supernova. So I told you at the very beginning about a supernova in 1987. This is a very important supernova, in part because it reinforced what Galileo saw: that the universe wasn't as we thought, that this was actually not a fixed heaven, that there was actually new stars — in this case it's a supernova, as we now know — that just came out of nowhere and moved with all the other stars on the sky. This was a disturbing idea for people that like the classical notion of the Earth at the center of the universe and an unchanging universe. It's the changes in the sky that have brought some of the greatest revolutions in the history of astronomy, and even mankind's understanding of our place in it.
And the reason why I have this quote from Hamlet is because there's a very interesting idea that you could read Hamlet as an allegory for the Copernican revolution, and you should Google that. It's a very interesting read. And it's believed, actually, that regardless of whether you read it that way or some other way, that in the first act, the first scene, when they talk about this interesting star, they're actually talking about Tycho's supernova: “Last night of all, when yond same star that's westward from the pole had made his course to illume that part of heaven where now it burns.” It's the change that tells us so much about the physics of what gives rise to that change. But the bigger picture, of course, is that that change tells us our place in the universe. And what I hope you've seen in the last hour or so is that we're about to hit a new revolution, where our role of discovering that change and understanding that change is itself changing. I'll end with that.
HOST: And Dr. Bloom, thank you very much. We'll take a few minutes to pause before we start question and answer period, and so that'll allow those of you who are unable to stay for the rest of the evening to exit. And again, you may use either the upper exits or the lower doors here, and the Academy's guest services staff will lead you out of the building, past the other event that is actually going on elsewhere in the building. And as you exit, we would like to remind you and invite you back to the next Dean lecture, which is on November 1st, again featuring Father George Coyne from the Vatican Observatory, talking about measuring the age of the universe. So we hope to see you again in November.
And Dr. Bloom, I guess the data deluge that you talked about, it's a case of there being too much of a good thing. Another example of that, I suppose, which we've been talking about in the planetarium, is the data coming in from the Kepler spacecraft as it searches for planets. Before launch, it was promoted that Kepler would discover hundreds and hundreds of new planetary systems. And, well, it did. In the first month, it discovered so much new information that NASA couldn't handle it all. And they had to ship out some of the data for other centers to process. So that's the kind of thing that you're referring to. There's just too much for any one organization or group to handle.
BLOOM: Yeah, I certainly didn't want to represent the data deluge is a bad thing, and I apologize if anyone came away with that understanding. It's a very good thing. It's just new to astronomers. We're not used to it. There's some small realms within astronomy that have had to deal with lots of data, but that's in a very restricted domain, and it's been very clear what to do with it. And we certainly don't pretend to be leading the scientific front in dealing with large amounts of data. I think probably the field that's been dealing with this a lot for a number of years is the physics field, especially the high-energy physics field, dealing with particle accelerators and the amount of data that comes from that. And the biology realm is absolutely afloat in data. So a number of different fields are wrestling with these generic problems in their own way.
The time domain, which is particularly applicable to Kepler, has its own special issues. And it's not just so much data. It's the fact that — not so much like Kepler, but for the stuff that I'm interested in scientifically — it's those objects that change, and we have to then get more data about them before they disappear, that we really have to understand how to get that classification done well and right immediately. Kepler has sort of a better regime, in the sense that if they've got an Earth-like planet, for instance, orbiting around a star like our sun, which would be one of the grand discoveries of Kepler, in fact, they have to wait for a while to confirm the discovery. So what they're looking for is an eclipse that happens every year, and then they want to see it happen a year later, and they want to see it happen a year later, and then they're going to wait to see it happen the fourth year, and then they're going to announce it.
So you can be sure — and if any of you saw Dimitar Sasselov's TED Talk — the Kepler people are playing their cards very close to their hand. You can be sure they've got Earth-like eclipses that they're being very careful that they wait until they have the confirmation observations. So yes, it's a lot of data, but their time domain is easier to deal with in the sense that they can just wait. In fact, they're supposed to wait. We're in the opposite realm of that, where we really can't wait.
HOST: We're going to take other questions from the audience now. Gary [inaudible] is going to take that side of the theater. I'll take this side, and we'll start and we'll alternate from one side to the other if we can, to try to make sure we cover the entire audience. So Gary, why don't we start with you?
AUDIENCE: Yes, good evening. Thank you for a very interesting talk. I was wondering, if you find 1.5 million new objects in one night, how do you name them? How do you go about — you can classify them, but how do you keep track?
BLOOM: I should send you some of the email threads we've had about that. So I should be clear that of those 1.5 million, only about one in a thousand of those are actually what we would consider astrophysically real. When you have a very bright star or you have a detector which is imperfect, subtracting an imperfect image from a slightly imperfect image, you get imperfections magnified. So most of the things that we actually discover in this 1.5 million are actually not real. And so the part of the machine learning that we're actually getting quite confident is actually working is our ability to have the machine just look through all of those 1.5 million and say, here are the thousand that you guys should really be looking at.
Then the question is, of those thousand, how do we name them? Well, most of those are asteroids, and we don't even name them. We let them fall on the floor. Don't tell NASA people that, but we do. We know about thousands of asteroids that we never name because we just don't have the infrastructure to do that. But of the objects — the variable stars that are real and the transients like a supernova which is real — we have a naming convention that goes a little bit like supernovae, where the first discovery of one of those objects within our data stream in a given year gets the classification A. So 09a was the first thing that we found in 2009, and then we go up to Z, and then we start back over at A, but we added on another A. So the 27th object was AA, and then we go up to AZ, and then we cycle around.
We have an internal database that actually lords over the naming, so that as somebody discovers something, either we have a human looking at data and they click on it — that's what we used to do. Now we have the machine codes that I've written and my group has written that actually essentially press that button and say, that's not only real, we think that's a supernova, and we think that's an interesting supernova. So those buttons are getting clicked, and effectively a database is getting populated with this whole naming convention. We're at something like 10uzl as of last night. So we've gone through two cycles of letters, and now we're all hoping by the end of the year we have four letters, but then we start worrying about all the four-letter words we're going to have to start putting in papers. So we're a little worried about that. Does that answer your question?
AUDIENCE: Yeah.
BLOOM: This is an internal naming convention, and Kepler, for instance, and Hubble, all these different groups have their own naming conventions for their own transients. There is no arbiter of what an object is actually called, apart from the International Astronomical Union, which gets to name comets, asteroids, and supernovae.
AUDIENCE: So the model we're used to is one big piece of equipment is designed to answer one big question or several big questions, like Kepler is going to find out if there are planets around other suns, et cetera. But this sounds like a different thing, this revolution you're talking about. So I'm wondering what properties of the universe, what known unknowns are you hoping that are going to be solved or constrained or something from all this statistical data that you're going to get? Are we just going to know better what percentages of stars or what? Are you going to learn something about the life cycle of stars, or galaxies, or the universe?
BLOOM: That's a great question. I would say, though, that while Kepler may discover the first Earth-mass planet around a star like our own in an Earth-like orbit in the Goldilocks zone, which would be this great, at least ticker-tape parade in New York City, the science that it's doing already now is something that the Kepler people themselves really didn't even envision, even though they had decades to think about what data streams they were going to get. The fact that every star is variable at the parts per million level or more is something completely unexpected. There is no such thing as a quiet star. So even though Kepler was made for essentially the express purpose of finding Earths, it's already having impacts across all scientific fields, in particular in the variable star field. It's already had a large number of interesting papers that have come out.
So that's one thing, but the bigger question that you had, of what do we hope to learn from this teeming universe of known knowns and known unknowns and unknown unknowns — I can't even wrap my head around the list of objects and inferences that we're hoping to get. But I'll try to name a few that are of interest to me. One is, if we're able to find all the RR Lyrae and Cepheid variables at some given brightness level — because we can know the distance just by looking at the period, which is, again, an incredible insight that Henrietta Swan Leavitt had more than a century ago — it means that we can actually make a 3D map of the entire Milky Way and beyond. So we'd be able to actually see very faint but real structures of former galaxies that were ripped apart by the tidal force of the Milky Way as they moved around.
The 3D tomography of the Milky Way — you'd think we know. Of course we know that there's a disk and we know that there's a bulge associated with it, but understanding all the stuff that's fallen in and the history of all the galaxies that we've ripped up is still out there for a discovery. And people today, just when they look in tiny little pencil beams on the sky, are finding structures that are much larger than the typical size that we think of when we think of the Milky Way. They're going way out into the dark matter halo. So that's one thing of interest.
The other thing which I hinted at that I'm quite interested in on the science front is trying to find these mini supernovae and associating them in a specific galaxy. That might not sound like much, but the gravity wave universe is something that we know must exist. There have already been Nobel Prizes given for it, so it must be right. It's more or less a basic tenet of general relativity that it should be gravity waves. And the way you should think about gravity waves is that you think about the analogy with light. If you take an electron and you wiggle it around — you accelerate an electron or a proton — you create light, you create electromagnetic energy. And that just propagates out at the speed of light if you're in a vacuum. If you take a mass of any mass — my hand — and I wiggle it around, I create ripples in the fabric of space-time, and those ripples propagate out at the speed of light.
We have indirect inferences at those things that exist, but we have never been able to make measurements of those directly. It turns out that neutron stars which smash into each other — which actually smash into each other because they're radiating gravitational waves, and so their orbit is decaying — those should be the first events that give rise to the direct measurement of gravitational waves. That'll also be a big ticker-tape parade, by the way. If the New York Times exists in three years from now — or four years from now — you'll probably see it.
But finding the electromagnetic signature of what happens when you smash two neutron stars together — we don't know what that's going to look like. We have an idea of maybe what we're going to find. Being able to find that concurrent event in the regime that we're used to looking at means that we can do some really interesting physics. It could tell us about what's happening in the cores of neutron stars. The densities of neutron stars are so high, we can't reproduce them on Earth. We don't have any good physical theory for what should be going on there. This could be — instead of big balls of neutrons, it could be some sort of soup of even subatomic particles like quarks. We don't know.
But we might actually, in detail — because we actually know the distance and the redshift, as it's called, to one of these objects — we could actually pull that out of the data. By making a measurement of these gravitational waves and knowing what we see in the electromagnetic regime might even allow us to make a very precise measurement of the expansion rate of the universe using just one of these events, that could rival all the work that's been done in the cosmic background for decades, with supernovae, with baryon oscillations — all these multi-billion-dollar facilities we might be able to best in just a few observations. So these are two. But if we had a couple more hours, I think I could keep on going. I don't know if that answers your question, though.
AUDIENCE: Yeah, I was wondering how much the utilization rate of telescopes has gone up since automation has come to this point, and are we now at the point that, beyond coming up with a couple high-quality telescopes, the data processing becomes a bottleneck versus the actual observing time?
BLOOM: So it's a good question. What's happening is that older telescopes and smaller telescopes are getting requisitioned because they're more or less obsolete unless we can do something really exciting and unique with them. So you have a meter-sized telescope that might have been the premier facility for a couple of universities or a couple colleges in some part of the U.S. 20, 30 years ago; it then morphed into sort of a teaching telescope. And now, in the era of constrained budgets, administrations look at that and say, why is that costing me this amount of money to run every year? The way you answer that is you say, well, I'll just automate it, and then it won't cost us much because we won't have to have people involved on a daily basis or nightly basis on that.
The concept of using old and smaller telescopes and roboticizing them is something that's catching on. And a number of groups, not just in the U.S., but throughout the world, are doing that. And the whole idea, of course, is that because they're small and they're old, they don't have a lot of demand for them. So by automating them, first of all, it removes a big barrier to entry. If I have to fly, or I have to send one of my graduate students to fly on the other side of the world, to use a one-meter telescope, I'm pretty hesitant these days. I'll really only use that sort of precious facility or precious money to go to very big facilities where the time is incredibly hard to come by. So robotization removes that barrier and then allows you to think big. If I don't have to send a graduate student or myself for a hundred nights to do one project, which is obviously a big hit on their social life, instead I can just essentially click a button on a website and I get a hundred nights’ worth of data to do a really big project. That's where some of the interesting innovations are happening these days.
So overall, what you wind up seeing in the landscape of all the available facilities in the United States is that the interests are shifting around. There's obviously more and more astronomers as a function of time. It's not itself a dying art. And what you notice, though, is that the pressure gets to the larger and larger scopes. And the smaller scopes, unless they've been roboticized or have some fancy instrument on them, really aren't getting so sought after. It's a really interesting thing to watch, over the last 10, 15 years, where things have been, and now understanding where they're heading. It's a time of great turmoil for our field. The big telescopes are sopping up a lot of the U.S. resources in terms of the financing of these large telescopes, in particular the Large Synoptic Survey Telescope. And then thinking about how to do the follow-up right means that when we look out at the landscape of our facilities, we see that we don't have the right instruments, for instance, to do this very efficiently. So it's driving kind of a renaissance in instrument building, getting the appropriate instruments on the back end of a small telescope to do interesting stuff.
AUDIENCE: Hi. It's fascinating to think about the opportunity that the data, the robotics, and the machine learning represent. It brings to mind two questions. One is, what do you think this means for the curriculum and the skill set of tomorrow's astronomy students? And then two, more philosophically, what does it mean to the relationship of man observing the night sky?
BLOOM: Those are great questions, and the first one in particular is one that I've been wringing my hands over for a while. In part, the National Science Foundation is getting involved in what we're doing and funding it because we've certainly said that we want to help the U.S. community get more educated. The astronomers need to get more educated in how to use these tools — not just use them, but actually innovate with them. And so this semester, actually, I started teaching two classes in computer science that was geared towards basically all physical scientists doing modern programming languages. And just today, before I came here, I was giving a lecture on databases, and I looked around at all these people that were taking this class and really excited about the program itself, and I said, how many of you used databases before? And two out of maybe 40 raise their hand. That gives you a great sense of this great disconnect between people that are going to be and already at the forefront of their own respective subfields, but don't have those very basic tools.
I didn't learn this stuff. I had to learn this stuff on the fly and on the job. And it's a big, huge systemic problem that we have. And universities are only just starting to figure this out. If you walk into the office of any astronomer, from undergraduate to professor, they are sitting in front of a computer. Most of them are not actually writing papers or making plots. They're writing computer codes. And that's not part of the core training of, certainly, a graduate student. There is no computer science program for a graduate student in astronomy. We've just started working on adding that and imbuing that into our training. So it's not that everything's lost, but there's going to be, I think, a large fraction of Ph.D.-trained astronomers who just don't have the skill set necessary to exploit what I've just presented to you here today.
And I certainly don't mean to represent that this is where everything is heading. This is where LSST is heading and forcing a part of astronomy. But that traditional role of flying out to a telescope, taking the data, looking at it yourself, while that itself is maybe on the outskirts and becoming more so, will always be there at some level. It just — we won't be able to do some of the forefront science that way anymore. So that's, let's say, my answer to your first question.
The second one, on the more philosophical question of mankind's connection to the universe: the obvious answer, of course, is we're losing a lot of that. I forget — something like 80% of the world's population can't see the Milky Way because they live in skies that are too bright. That kind of immediate connection, which you don't need to have a telescope to view, has more or less disappeared already and is getting even worse as a function of time. But getting people connected to the skies is not easy. Obviously you guys are all here, so you're on the right track, and this room attracts a large number of people — I don't know how many you get every year — but obviously people remain excited about it. Understanding the heavens is something that I'd be hard-pressed to believe that we would turn our backs on as humans. But like many of the physical sciences that don't have obvious connections to our everyday lives, it becomes hard to make the case that we need the next billion-dollar facility.
I think we can do that. There's lots of practical, relevant tools that we're able to provide for the community, and have in the past — in particular, the CCD. Every one of you who has a cell phone has a camera that has a CCD in it. That was largely driven because of the needs of astronomy. So I don't know. I could also go on this topic for a while. Maybe I should ask you what your worries are, given what I've just said and what you've just seen, on that second topic. I'm turning it back to you. We're going to have a conversation.
AUDIENCE: Well, I don't know that it's a worry, so to speak, but when you think about when you stand there — and how you reference observing the Milky Way — when you have the opportunity to stand outside, see that, and even in San Francisco here you don't have that opportunity unless you head up to Mount Tam or somewhere, it is quite grounding in where you are here in this vast universe. And I guess throughout your talk I kept going back to the “Pale Blue Dot,” Carl Sagan's famous talk that he gave at Cornell back in the mid-90s, and how that had such a strong impact on the environmental movement as well. So I think more and more, what we're doing and observing in the night sky is reaching into other sciences, and it's coming together. So I don't know that it's a worry. It's just, I think it's important for all of us to share this passion and interest. So, more of a statement. Thank you.
AUDIENCE: Question here: talking about all that data — your team's got tons of data, and everybody's getting tons of data, too much to deal with in some capacity, in some ways, that you're trying to set up this federation and get these machines intelligent. How much is — if I'm studying asteroids, and I hear you got a bunch of asteroids you don't particularly care about, do I get the opportunity to data-mine what you've got and sort of meld the data? Or how much does the sharing happen, or is everybody just so full of data that they don't want to bother?
BLOOM: No, I think that's actually an important component of where this field is at now. There are very few data streams that are so precious that no one's willing to share them. The Kepler people will not share their Earth data until they publish it, right? But after it's published, everyone's going to get to look at it. It'll be on some website, you can pull over the stuff, and you can play with it all you want. That's been mandated by NASA, that they do that. In the context of the Palomar Transient Factory, I said we have all these asteroids that are more or less falling on the floor. If you're interested in asteroids, you send me an email and I say, that sounds good with me. And then I send an email to the principal investigator of the project. He'll say, that sounds good with him. And then we'll say, can you supply any people that can actually help us build the software so we can ship off that data to you and run the codes that you want on our own machines, or just even get that infrastructure running?
In the end, it comes down to people doing work. And while I've talked about the abstraction of humans in the role of discovery and inference, that abstraction itself has to be implemented by people who are aiming towards that. But this concept of data sharing — you're right. Because there's so much data, this older notion of, well, I've got my precious image of the sky, you can't have it till I've milked it for all it's worth — that whole psychology or sociology of the field is really changing, just because I feel guilty for having stuff on my disk that you can't look at. It's the guilt. Maybe guilt is a great motivator. So if you truly are interested in asteroids, send me an email.
HOST: And if there are no further questions, Dr. Bloom, thank you very much.
BLOOM: Thanks.
HOST: Thank you for listening. For more lectures, please visit us on iTunes or at www.calacademy.org.