Transcript: Bobby Kasthuri & Brain Mapping – Episode #2

okay welcome to our podcast my name is Cory Washington and this is my co-host Steve Hsu and we’re gonna tell you a little bit about how we hope our next episode should be going and some of the plans we’ve got to the show but we’d like to lay out kind of a general philosophy to start now one of the things we really want to do in this show is always present the opposing point of view but in a respectful and informed way we always feel that this stronger sense a stronger understanding of a particular subject emerges when you hear both sides of it and so many sides the case more many sides yeah there may be more than just two and so there may be times when we have a guest on who’s controversial and neither Cory nor I agree with what that person’s thesis is but we’ll listen respectfully and then we’ll try to raise what we think are the most the most salient objections similarly we might have a guest who actually we don’t have any objection to what they’re saying we’re just fascinated by them and want to hear what they have to say but nevertheless we will try as a duty to the audience to still raise some points against their perspective so then everybody emerges with a broader sense of what’s going on I think that’s gonna be true for a lot of science shows that come up you know first of all we meet talking to people who we’re not experts at area and so don’t strong positions about their view and also we just may agree with their science but you know science works best when you have kind of sort of a critical eye at it and you can look at its assumptions and some of the claims are being made in a critical way so we really want to kind of have kind of a good-natured a kind of you know sharp sharp yeah discussion adversarial discussion sometimes and we won’t just be talking about science we plan to have people on who are philosophers maybe we want to talk about politics who want to talk about UFOs that’s right whatever it is by god god yes so we’re very excited about our first guess this will be the first interview that we’ve done our guest today is Bobby Narayan a neuro scientist from Argonne National Labs and the University of Chicago I’ve known Bobby for several years and we’re kind of friendly at least up until this interview and so I’m gonna be the main interlocutor Cory is welcome to interject hurl invective at us disagree with anything that we say but I’ll try to leave the interview with some specific topics that we want to cover awesome and let’s start with your Bible fee and your career I’m gonna do my best just saying a few things about it and would love if you would elaborate on some music I was about to say about you so Bobby I you were born in the United States is that right no in India Coon or Indian oh and when did you come to the US when I was six years old five years old in 1980 and where did you grow up I grew up in a small most of our time I grew up in a small village called Coon or India in the southeast of India and then spent a lot of my time in Madras India or which is called Chennai India now which is where my grandparents but how about in the u.s. in the u.s. I grew up in the great state of New Jersey yes or from New Jersey that I am from India okay so you grew up in New Jersey and somehow found your way not probably not far from where you grew up to Princeton University where you did my undergraduate degree um and what was your major I had an odd measure this will come up over the I was we talk about my career because I’m on technically my fifth or sixth career currently when I was in in college I was really interested in the space of scientific public policy so I majored in molecular biology and had a second major it’s something called the Woodrow Wilson School for public and international policy which is efforts very good and so subsequent to Princeton you were a I believe a Rhodes Scholar and you did your PhD at Oxford am i right about that correct and was it at that point that you became a neuroscientist so there were a few jobs potential jobs in between in between public policy and where I am now I really thought I would make a lot of money in healthcare so I started medical school to try to just get my understanding of Medicine so I could make a lot of money in healthcare it was pretty obvious after a year that that’s not for me so I decided oh well maybe basic research is the right way to go so I had already done a year of medicine and just randomly applied for this scholarship but then I got and then because I got it I decided to become a scientist great and and I think your MD is from Washington University in st. Louis is that right correct great and so then subsequent to your PhD you were you followed the typical academic research track you were a postdoctoral researcher at let me see if I have this right Boston University and Harvard I was at Harvard do my postdoc and I was briefly a professor at BU got it okay and in what department puppy it be sorry at Harvard almost all the departments are variants of anatomy and neuroscience neurobiology etc I think at Harvard it’s called neurobiology and it bu is called anatomy in neurobiology great and it was there that you really got into brain mapping is that right mm-hmm and I think you have a patent on some kind of device that does fine slicing of brains and then floats the slice on water so that it can be I guess interrogated by is it x-rays electrons electrons electron microscopy it’s like a tiny conveyor belt is that right it is literally a tiny conveyor belt it’s a it is as Rube Goldberg esque as you can imagine an invention and we could discuss this but like making a tiny conveyor belt and adding it to the normal workflow of electron microscopy is the reason why I have an academic job right now so they help how big Bobby is this conveyor belt a conveyor belt this about in fact we modeled it very old audio tapes so it’s about eight to ten millimeter and it you know in lengths or so and the reason we did it was that like one of the things and Steve you mind if I push fast or should I wait no no well I think this is sort of along the lines of where we’re heading anyway just quickly maybe just quickly say so it’s eight millimeters and then the thickness of the slice is roughly something on the order of each brain slice is on the order of 40 to 50 nanometers in thickness so Praetorian another way to say that is 400 to 500 atoms if you imagine that an angstrom is about the size of an atom and a ton of those a nanometer you get to something like that and the reason we have to cut these things so thin is that in a in a human brain there’s something like a hundred billion neurons and each of them make ten thousand connections with each other so that means the number of connections connections between neurons in a human brain is on-order quadrillion or hundreds of trillions if you multiply those two numbers together the only way you can fit that number of anything into the volume into the volume of a cranium which you will into the volume of a human skull is to make each of those elements extremely small smaller actually than the wavelength of visible lights so a neuronal connection is actually smaller than a wavelength of visible light so you can’t use kind of standard optical microscopes to map a brain you have to use electron microscopy the downside of using electron microscopy is that the electrons don’t penetrate very deep into your sample they scatter really fast so the only way you could map a brain is to cut it into a whole bunch of thin slices each 50 nanometers or so and you have to be able to collect something like fifty thousand slices each 140 nanometers in a row and that seemed physically impossible until a collaborator at Harvard we’re like well maybe a tiny conveyor belt solves the problem great and and so the in terms of the actual devices is it Zeiss sells a device based on your work around the ground and so how many of these how many labs have these around the world so I would say there are about fifteen or sixteen labs in the US that have this device and the reason I I think more people don’t have this device is that the device only solves half the problem which is that once you collect a ball you even something a millimeter tube so just to give you an idea that’s smaller than a grain of sand a grade of dust if you go if I did it a millimeter cube of a mouse brain I am resolution it would be something like a million gigabytes of data and so the conveyor belt gives you the ability to capture that data to collect that data but we’re still struggling with what to do with how to analyze right the situation you’re describing is pretty common across multiple Sciences where the instruments that are relatively inexpensive now can be used to collect far more data than we really actually know what to do with correct and in a lot of fields like Steve this apparently and you know CERN was a pretty good instrument externally so and sorry but an expensive one yes worth it and you know I heard that at CERN they collect similar numbers they collect thousand terabytes a day it’s not unheard of for surd yes but what they wind up doing is throwing away a lot of that data as they’re collecting yeah it’s even worse than that because if the hardware level they design vetoes that that choose not to record big chunks most of the data actually it’s only piping the number you gave was the data that’s piped out and that’s exactly role hardware level filters so generating far more data than that actually but yeah I know and and it’s because physicists are like are fundamentally smarter we can remind me to tell you about physics and neuroscience but they know exactly what to look for to understand in their particular system something that’s interesting about neuro sciences we’re not really sure what the right metric we want to measure to understand a map of the brain so we’re forced to kind of collect all the data now and see which parts of it provide understanding and maybe the future brains that we collect will be able to adopt the CERN philosophy where we’ll be able to throw away a bunch of this data so in my notes for this section of our discussion I want I what I had written was you know describe the research for a general audience but then also for an expert and I feel like we’ve kind of done that a little bit but let me turn to my Ombudsman quarry and say if we have have we described what he’s doing in a general enough way and also then what would an expert want to know more about that he does so I don’t want to rain on the parade a little bit but I think it’s actually not clear the audience what Bobby’s conveyor belt does or how it fits into his excellent little research so get him doing evaporate on there so Bobby we don’t have visuals for this talk right I think can you try to explain basically how a slice house slicer is gonna work and how they’re gonna crimp in sections and then what happens to them not like ham so it’s a lot like ham action exactly right so give people a visual image of where you’re of you know of the size the brains you’re cutting the size of slices that are coming off and the way your conveyor belt sits in that process ah so let me start off with what the constraints are and then I’ll tell you how the solution matches those constraints so the very first thing I said was that we have to use electron microscopy because the density of all the things the size of all those things in the brain is requires it what that means is we have to cut the brain into a whole bunch of little thin sections in a row like cutting slices of a salami or a sausage or etc and one thing to say is that it’s very hard to cut tissue in an exacting way because it’s wet it’s weird it squirts around etc so scientists particularly neuroscientists have developed ways to prepare a brain usually a dead brain a fixed frame we stain it with a bunch of metals that the electron microscope can see and then we embed it in plastic we literally turn the whole brain into a piece of plastic and if you can imagine that piece of plastic attached to a lever arm can you guys imagine that in your mind the lever arm is going up and down and it’s advancing in X nanometer steps right except right and that’s what we call the sample right next to the sample Steve if you could draw make a little fist for the sample yeah and right next to it is a knife and that knife is made of diamond literally it’s a piece of diamond sharpened down as that block advances sections get cut those sections then float on water and then at the end of that water trough is that conveyor belt so the section somebody so messed up because this is the problem you saw I think many people have done slice slices in their lifetime have had to deal with this previously you know you basically have a little slice come off and it falls into kind of some sort of solution you’d pick it up by hand I have Weiser put it slice on a slide right and you do this for repeatedly happen time you mangle the slice is you get a little bit and so you wanted to solve this problem of trying to get thousands of slices all done perfectly without you know massive carpal tunnel syndrome exactly and in fact you know the human record for the most number of slices ever collected is about a thousand or so and even worse the people who do it are artisans the people who have the patience and motor skills to pick up section after section after section that’s actually a dying form it’s a lost art form people aren’t trained in it anymore so the idea that we want we want to produce something that was robust that we could distribute to labs all over the world and that could scale so it turned out the same way John Henry losses to the steam machine I suspect these artisans are winding up losing to the to a conveyor belt and just like John Henry they’re super upset so I think that gets people picture of very good your invention yeah I should say that we’ll probably put in the show notes links to technical seminar that you’ve given I think you gave one spirit Emma to a UFO which has lots of slides and visuals and then even like I think I’ve seen on the web little videos of your machine there are an animation of your machine in action and that’ll make it clear to people who still are a little bit confused about what it does how much money they made off your machine currently because Harvard you know essentially owns the patents I have made zero dollars that’s sad but okay but dice is probably sold like got a no mill tens of millions of dollars worth of these machines right yes yes someday I hope to make dozens of dollars but currently this is neither here nor there the first set of instruments they sold were prototype you know they went at cetera etc etc so they felt like they shouldn’t share the money for those first X set of instruments because they needed to use it to keep revamping the ESEs so one of the topics further down my list is scientists are underpaid and society under invests in scientific research but we will we won’t get we won’t branch off into that just yet yeah let’s even the research and so Cory you’re an expert in neuroscience what’s a thing about what Bobby does that would be that an expert might like to hear more about so I guess I’d be very interested to find out you know how the techniques you’re using right now Bobby are advanced is beyond what went before you know memory talking about how physicists throw away a lot of data yeah so it’s pretty interesting is that early neuroscientists did this almost by accident was very very beneficial so as you know the early stains right done by qahal you have in the Golgi man that only stained about one percent of the cells maybe less makes have no idea yeah exact precise see but as a result you could actually see individual neurons yeah and and qahal is able to see effectively synapses right so he because the stain was so in fishing it was sparse enough to get a nice picture of it which I thought was really fast things that’s what’s really why it worked but since then has been a whole train of research of ways of mapping the brain and you’re doing work right now which is using really cutting-edge techniques so how would you describe your work as an advance of what went before what was before there before you came in and what do you definites new yeah so I I think it’s a great point Cori I think the advantage and the the thing wrong with kaha by the way the guy that Cori mentions is a guy named Santiago Ramona call who won the Nobel Prize probably in the early 20th century for really changing one fundamental view of the brain which is that it’s made of cells the brain is made of cells called neurons neurons have specialized parts to them dendrites and axons and that they connect with each other over empty space called synapses or connections what’s amazing is even actually see a single connection because he had the sparse labeling and most of the times you don’t know who that cell is connecting to because 99.9% of the cells are not lis if you will in the data set and what most people do is then they go from animal to animal to animal so if only one of a hundred cells was labeled you might imagine well if I do it over a hundred animals or or some 500 animals you should be able to recapitulate and it’s and labeling is random etc you should be able to synthesize potentially all of that back together maybe into one braid and not go through all the hassle of the cutting and millions of terabytes of data I’m claiming that we have to do I think the main issue with that is that fundamentally brains especially mammalian brains are not identical from E from one brain to the next frame to the next if I had the same identical brain sometimes people think in vertebrate animals like flies have the exact same brain again and again and again and therefore it’s possible to sparsely sample the same identical net multiple times and make an inference about how that network happens I think in more complicated brains like mammalian brains the who you connect with is actually dependent on who else they connect the system itself is it’s not identical that system is designed to come to sort of for the history of that brain and that history of that brain makes sense when you look at the network not when you make when you look at individual neurons over many brains that have different histories essentially different connection matrices included all that back together yes another way I would say if you don’t mind is that implicit in the word circuit a neural circuit which is the collection of neurons connected together to make a behavior is this idea of a circle it’s hard to say the word circuit without imagining the word circle so at some point you would imagine that and it should be true that you could go from a neuron to another neuron to another neuron to another neuron and back to yourself and in reality no one has ever visualized a single circuit in neuroscience that is responsible for a behavior which is do proxies of it we claim it’s circuitry but if you got somebody the one time perhaps it’s ever been done is a very small animal called C elegans that at 302 neurons and someone mapped all of its connections by hand manually like you were saying these artisans who pulled C elegans section after C elegans section and when you look at that wiring diagram how that circuitry connects with each other it’s way more complicated and has motifs in it circuit motifs repeating things that would be very hard to infer just by looking from single neurons from animal to animal so the last way to say this and Steve notes I think this way I’m always surprised by why we haven’t achieved more in neuroscience the annual Society for Neuroscience is I don’t know fifty five thousand people a year it’s held once a year I think 60 thousand it’s the single largest convention for science I think the cardiologists used to have more than us but we beat them a recently the NIH budget for Neuroscience is something like a couple of billion dollars a year and only about half the people go to neuroscience so if you have a hundred thousand smart dedicated human beings spending billions of the government’s money a year and you multiply that over 15 20 years you might ask well what the hell up you guys discovered but very hard problem right and I guess it specifically but possibly is a very hard problem brain is most complicated thing we know of in the universe right to say that is job security just in my opinion but let me push this tiny bit more Steve if you don’t mind another version is that of course it’s a complicated system but you know the immune system is pretty complicated the heart is pretty complicated the kidneys are pretty complicated we have gone so far and we can make artificial versions of almost all of those things that I’m describing to you we’re not even close to that to a brain one version my version is that because neuroscience is very rich and dataport most neuroscientists whether they work in a lab or they work with math are actually theoretical neuroscientists they’re working on very sophisticated hypotheses very intimate theories of the brain how individual molecules can influence memory and behavior is like the standard routine for a neuroscience given that we don’t even know when 99% of the cells do who are based on this sparse staining and she seems like you know the cart before the horse sort of thing I don’t know do this you guys are so I think you’ve led precisely into the next point that I wanted to get to which is a statement that I wanted you to react to so let me let me read the statement and then we can discuss a little bit more about how the statements related to it the point you were just making the same in is tool builders perhaps secretly are the real drivers of scientific progress broad ideas by themselves can be overrated so actually love to hear both of you react to that but why don’t you go first Bobbi okay so I’ll react it’s I think this is pretty telling and did you do this on purpose T I can’t agree with a statement more that misstatement that you just made when I give my talks I do this thing which I can put on a podcast if you want which is I make a list of the Nobel prizes over the last hundred years that have been received for a specific idea and the vast majority of them might say a hundred percent of them but that’s a little probably too much because that person had access to a tool or technology that nobody else did and in fact to go back to my hero qahal we were discussing earlier that guy had access to two things that the vast majority of neuroscientists didn’t have access to he had access to this stain that Corey was telling us about that only labels a very small percentage of all the neurons in event that is frenemy Camillo Golgi and he he was also one of the first people to use a compound microscope with glass diffraction optics to look at brain slices now there are things that qahal discovered using those tools but I don’t think I could discard not that smart right but like I would say 70 or 80 percent of the things that he discovered I could have matched if I had access to technology that nobody else it’s my reaction to the statement is I think it’s really a kind of false dichotomy you know between technology on the one hand and ideas in the other I think is my reading of neuroscience is your neuroscience is that whatever you have these kind of significant advances it’s because you have an idea wedded to a technology someone had a are’ and they had a way of testing the theory right which may have been novel but sometimes they had a theory and not super novel technology but I don’t think if you have them separated you know if you have ideas about technology your pure philosophy if you’ve got technology with no particular ideas that doesn’t lead anywhere any it lead anywhere either so I think this is this that’s one of my problems a lot of neuroscience I think it’s a lot of cut technology driven research well that in particular theory behind it you know when I was in grad school the two-photon microscope is becoming very popular and now they’re they’re fairly common uh but you know there was our lab was one of the few that had them and people were doing a lot of experiments with a two-photon but it wasn’t clear what these experiments were supposed to show you could get some pretty pictures and you can see some synapses I’m not sure they were never seen before we saw before you and we’re brilliant way they’d had before but there’s no outcome to that as far as I could tell so that struck me as a case where technology was not what into ideas yes and I think you know it’s I think I think the key things to be successful I think we’ve really this Bob you have to have like kind of both ideas I think many great scientists are actually kind of philosophers at the same time they’ve got a way of making these ideas because there’s practical and testable yes I mean qahal I think is quite interesting you know I remember looking back at call and I was actually in a reading group um early part of grad school one person pointed out of vodka huh is not only did he have the Golgi stain he was actually probably the best practitioner the Golgi stain he’s better than Golgi at the Golgi stain yes and so his his uh stained Rex he so good you go you could all see the synapses you couldn’t see them gold cheese right and a little bit hard to see but you could see the McCall’s and not in gold use when he looked at me so you probably see why he got the Golgi didn’t Goldie’s were just bit too fuzzy around the connections but girls were picture-perfect and something of yeah it really caused a lot of technical expertise to uh to do what he did and that’s what I think led to out of his discoveries so Steve do you mind if I push back one more time yeah you tied the flop the philosophical question back into the technological question go ahead the reason I think technology is the answer is because of philosophy of science so so let me push back or not push back let me try let’s say there’s two main ways to imagine how to do science the first is this way actually both ways were published quite CIPA close to each other the 1950s the first way is if you think of pop and how popper karl popper one of the great philosophers of science so well the way science advances is by falsification I have a theory I try to falsify that theory with data and of course technology can help what that falls vacation but I really need that theory first before I can invent the technology to help falsify I think the alternative view of the philosophy of science is this kind of Thomas Kuhn view a science and Thomas Kuhn I think wrote shortly after popper did set wrote this book called the structure of scientific revolutions and in this book cocoon makes this argument he’s actually the first person I think to make this these words paradigm shift he comes up with this idea of a paradigm shift and his view of science how science progresses is a lot different than how popper used in the kuhnian view the existing scientific world for socio law the sociology of it does everything it can to prevent the destruction of its world view of science there’s an established field the two examples the interesting example is this but Copernican versus patola me view of how the planets move around the world they move around the earth or do they move around the Sun so there’s a worldview for a long time right the planets move around the earth and lots of people collect data consistent with that of course it was rudimentary data at the time and then what happens is somebody collects a piece of data that is not consistent with this or or throw some dirt into this worldview which is that it turns out that for all the planets to move around the earth some of the planets have to go backwards or or do weird loops etc right and that little dirt in the oyster is what becomes a pearl of a new scientific theory another version of that is blackbody radiation I imagine Steve knows this better than anybody who’s these weird little facts that don’t make sense with the worldview that are surprises and then you’ve knocked down that worldview and build up without that’s not a gradual review of science constantly if you’re interested in that second worldview I would argue that the way to do that is to invent technology that reveals a surprise and and that the wheel chances are these surprises in science that knockdown views so even though the two-photon microscope invented by Winfrey Deng who then went on to invent a lot of these connectomics tools that I was telling you about so that’s interesting the two-photon microscope I think hasn’t been proven yet because it yet hasn’t discovered the surprise it’s really good and validating or in validating hypotheses well we want our datasets where the surprise lives and my argument may need to go back to earlier quarry the reason I want to collect all this data in a way that nobody has seen it before is somewhere in there is the surprise now that’s not good for granting agencies and others like to come up with reasons for it but the reality is I think that’s where technology helps it is this thing that produces surprises that often winds up being the truth that’s my last piece sorry I don’t get too deeply yeah I think there’s I think they I think the pop hare in view is some part to science work that way neuroscience not so much yeah I think poppers I think mostly thinking about physics actually and the chemistry to some extent Koons right this certain certain certain anomalous phenomena do arise lead you to get out sort of change things could it is a very strange kind of almost anti-realist view where he thought these paradigms are almost self sealed right and you couldn’t talk about falsification at all you’re just moving one paradigm to another and it’s not that one was more true than the other just like a different idea system it’s almost Koons almost postmodern I mean in that sense maybe the people have adapted him to post-modernism it’s possibly even in him right he says things that are a little bit like that um but I think each of these both have something to say and but never signed to think if you look at some of the developments and pretty interesting right if you look at you know you know for instance we know people discover it you know the currents but hodgkin-huxley it did there pay there are voltage clamp experiments they didn’t have a kind of new technique but they also had a certain theory about you know you want to be able to control from voltage rumble the currents sure and out of that you know again I’ll that debate they made a mathematical model that all but did not postulate channels as physical entities effectively had you know sodium and potassium channels in it so you know they pretty rapidly develop this theory based upon certain assumptions which drove their technologies that then led to a huge range of other experiments right so the idea the general idea first a technology much more self specific idea after that and then you know correct since modern neuroscience took off but yeah I don’t hijack Steve’s narrative here no that’s okay III don’t have much to object to in anything that either of you said I would just say and I think most scientists would agree that technologies are important and ideas are important I mean the latter maybe just because the space of possible hypotheses is so big that you couldn’t just randomly generate hypotheses and test them you would never finish so you need some sense of taste or some way of narrowing the set of ideas that are worth testing that are worth spending all this time developing technologies to test so I don’t think there’s too much super controversial in that sense Oh Steve can I try to make a controversial okay well if you insist go ahead okay I always insist I’m assuming that’s why you invited me to a podcast so go ahead go ahead here’s the last piece there is no person in know very few people in neuroscience practice either pop Aryan or kuhnian science by that I mean I have never met a neuroscientist especially the more famous they get who is interested in disproving their theory of the brain most neuroscience and inquiry I can see you you can’t see me so you don’t see me smiling we’ve gone through Theory after Theory long-term potentiation is the basis of memory spike timing-dependent plasticity is the basis of memory retrograde nitric oxide so crazy to think that was the molecule of the year is the basis of memory all these scientists who come up with a theory don’t want that theory to die they want their theory to actually be correct is this true scientist human beings in general Bobby nobody wants to third anything falsified yes I think it’s a matter of degree though yeah I think the best say physicists wouldn’t you know defend irrationally defend their pet idea yes but I think the incentives are what Corey says I mean almost everybody does actually defend their idea a little bit too long yeah maybe all the way to the grave yes that’s that’s the worst case scenario but certainly a little bit more than then an uninterested observer who hadn’t written the key papers in that for that particular hypothesis would defend the work so IIIi agree with that I don’t disagree with that well then maybe I’ll try something controversial later okay let me let me go on to another statement which I want you both to react to which maybe it is controversy maybe it’s not but it’s sort of my view of what’s happening right now a little bit so let me start with AI so in AI we’ve made a lot of progress using things going deep general nets and deep neural nets really are in a way a kind of crude cartoon version of how the human brain might operate in the sense that there are layers and there are connections between the layers and there are things you could even call neurons or synapses but in case individual nodes of the neural net affect other nodes and there’s a certain strength by which they affect other nodes so there’s a connectivity and then there’s a strength of connections within that network connectivity and we have examples now of really successful deep learning really successful neural nets so for example recently a company called deep mind trained a neural net called alphago we shout plays the best human players and go yes and go is a game which has been played seriously by humans for thousands of years and most computers computer scientists thought it was going to be a long time before computers could beat humans that go but now they’ve just in a very short time totally eclipsed humans at the game of go now unlike your problem the trouble the the huge effort you have to go to to image the inside of say a mouse brain or something like that yes these scientists are engineers at deep mind can just open up the neural net which plays go they can open up alphago and they know every thing about it they know it’s a connectivity they know how the connections work they even wrote the they wrote the way in which information is processed by that network so there’s nothing unknown to them about alphago well let me however push back on that Steve because isn’t it I mean this was sort of so many these connections were just were done by simply cyclical learning of previous games of go so so the many many connections in this neural network and it’s not clear that’s you know how it functions they know that it gets the right answer but in fact these C scientists scientists actually do ablation experiments on neural networks so we have a function well it’s R so it’s so this is this is the point I’m trying to make actually so so from a descriptive viewpoint there’s nothing further to be learned I mean it is represented fully in silico so they’ve been actual neural have the anatomy anatomy is fully known of this neural network ok so it’s as if as if Bobby’s program had been carried through to completion so you had a complete one angstrom level resolution map of the mouse ok and even more than that you had really accurate dynamical models of how like a charge buildup here would do something here so so they don’t know everything like given initials given the state of alphago at time T they can tell you perfectly what the state will be at time T plus 1 ok no question no question now we’re in that huge body of information it’s millions of connections Strength’s millions of numbers right that specify the neural net we’re in that is an understanding of the game go yes so you can have an example where and so this is related to the statement that our brain might be too complex to be understood by our brain yeah what I mean by that is even if you gave me a perfect anatomical description and even dynamical description of how our brain processes information or shifts electrons around we might never actually locate in the brain the exact place where it knows that if it rains at night the lawn is wet right that’s a thing that we think most humans can figure out that little syllogism but we may not figure out mechanically how it’s represented by all the complexity of information we get from the anatomical description of the brain and that’s kind of clear now from what’s happening in deep learning and deep neural nets that there is no way we can really open up alphago and figure out the game of go at least not easily now Cory actually already leapt ahead to ablation experiments where you you say like well let’s suppose I blow or randomized some sub chunk of connections what does it do to gameplay like does nothing like start making bad moves but only bad moves in the opening or only bad moves in this subset of situations so you can’t probe it but that in and of itself is a is that maybe perhaps hundred year project to figure out how does alpha how is the game of Go representative in alpha go so maybe just react to that so I would agree Steve I think the assumption is why would anyone imagine we would understand a brain anyway meaning I would say in another way what do we actually understand about biological systems that week that isn’t essentially reverse engineering or an or an engineering principle couch does understand now this is going to dive us possibly deep into what the word understanding means whether I understand what understanding means we get into these weird loops etc but this doesn’t surprise me at all it would be similar to saying do you expect a dog to understand regular physics no of course brains have limits and capacities and and I would argue we barely know anything about but to say something a little bit controversial if I were to go to the annual meeting of the neuroscience whatever it’s called Association and poor people have them react to that statement how many of them do you think would have a relatively deep understanding of what that sentence might be in our brain is too complex to be understood by our brain well can I jump it here I think partly it’s just the question is a little bit kind of grossly formed right there do you understand the brain is a whole level I think doesn’t make a lot of sense I think and this is true biological phenomena and they are oddly understood by neuroscientists I don’t know I haven’t polled people I think whether you lived in the community and I think people people work on tiny tiny phenomenon this is true biological phenomena you know here’s an analogy suppose someone says you know how does the natural world work right how does the earth work you know I see these animals you know they get some food and then they go mate and have some babies you know those things dying it’s just it’s also complicating well you just give a nice compressed I never I never I never dentist phenomena exactly exactly exactly that’s funny what a Baldy works neuro Center give tiny theories generalizations that don’t go very far apply to a very small region maybe there is a similar phenomena in another animal or some uh phenomena nother part of the brain but you bet it’s a micro theories and that’s how it works then you have a global theory the brain I think is fantasy I agree maybe simply portal just could have it but very good I’m pro reverse engineering I mean at some level I can make the case that I understand brains to a pretty good approximation already if I didn’t understand brains how could I live my normal life I make predictions which is one version of understanding about brains around me all the time not as a neuroscientist but as a human being living and working in the world I can make predictions now about if I started talking politics with you guys or sports or whatever I as a human being I’m really good at making predictions about brains the question is whether whether from a bottom up I can make predictions uh whether taking these small facts exactly like what we’re saying and turning them into broad general theories I’m not sure that’s ever worked so Bobby let me let me in reference to this particular topic that were on let me reference a little short story that I wrote on my blog character in the story so you’re working in your lab with your slicer and Dicer and electron like oh you’re working very hard and you’ve got out like a thousand grad students slaved in harnesses doing whatever they need to do and I come in down from down the hall carrying a little box and I said hey Bobby look what my Foreman he works at deep brain now and this is the very first artificially intelligent device it actually has passed the Turing test it passed the advanced Turing test passed Turing test five and look it’s a little neural network that they trained in a virtual world so they actually have the neural net and they trained it but then they instantiated in this this little blobby silico things we’re going to have to stop you need to explain what’s the Turing test what seemed an eternity during test 5 just briefly I know bud have a lot of time Turing test is a functional way to determine whether something has quote real intelligence or are you know that an artificial intelligences is really passed some milestone and the way the test is conducted is by just having people probe the artificial brain and ask it questions see if it can learn things and if it can fool them into thinking that it’s a human then in some sense it’s got at least sort of human level intelligence anyway the little story I run into your lab and I’m carrying this box and the box is a substantiation of a neural net that was trained maybe inside a computer but it now can do all kinds of things that the human brain can be maybe it’s even smarter than the typical human and then in the story and to poke fun at you juror that is Bobby in the story rushing over grabbing the box out of my hands because you want to immediately start imaging it yes conditions that are going on inside the box and I sort of weekly try to explain you know Bobby we’ve got that all like I could but but you were just really obsessed that you want to measure all the connections between all the little sub pieces because after all this is a super intelligent alien being that we’ve just discovered what do you think of my little story is that in fact what you would do Bobby sorry which part yeah if you had the brain would you be inclined to try to image it if there is a data section where all the connections were anyway fair point I mean if Steve said here’s this box and here’s what we call the wiring diagram which is how every neuron connects to every other neuron I’m generally not a trustworthy person sorry I’m not a trusting person I think that was a Freudian slip I would I would think about whether I trusted Steve or not and if I didn’t I would redo it but if I trusted him I don’t see any reason to go to redo the wiring guy that part of the story was an exaggeration because if really understood that we had actually produced the thing through artificial means and therefore had the wiring diagram stored somewhere else he wouldn’t grab the physical but it was meant to poke fun at this perhaps what could be interpreted it I think maybe some other neuroscientists interpret as an obsession with measuring deaths from the gain from which is uncertain yes in fact ok another thing that Steve has called me in the past is kind of the ultimate stamp collector that I’m literally interested in collecting every stamp of every connection of every neuron in a brain and there were times where that was useful you know doc Darwin cataloging all the things he catalogued with the Galapagos is stamp collecting I think it’s a very fair question to ask is there a return on this I have to collect every stamp what if I just like the 30%
of the stamps would I be ok I don’t have good answers to these questions because it hasn’t been even partially done yet I think my initial reaction when I learned about the the really you know the big advances in brain mapping and how far we had come was wow that’s amazing we should just do it in terms of the total neuroscience budget or scientific budget it’s it’s not such a big fraction and so why not just get a decent you know X level resolution map up the brain let’s do it Cori has sort of argued that it’s kind of clear that we don’t really need that right now we’re not going to get all that much out of it maybe they’ll be surprised that people didn’t anticipate but people can’t point to right now a high probability game that we’ll get from such a detailed map is that fair yeah I just I’d say let’s look at history right next he got some pretty good small maps of different brains and there’s no doubt we’ve learned some things from Anatomy but you know again I think you know Anatomy works in conjunction with physiology right you first learn where connections are then you probe those connections by measuring electrically from them and then you may begin to get understanding how certain set of cells reacts to stimuli but that’s where I think the eventually person where I see the value in anatomy said of certain going whole hog and mapping everything you make you basically make it a part of a kind of broader theory development program are you there are where things connect you learn how those connections behave they try to understand a small region of the brain rather than shorts going on for so that’s that’s kind of very different from your approach pop you like to hear your reaction to that yeah all skill neuroscience sorry Cory I didn’t hear the last part I guess I’m really giving you a kind of uh as you use the kind of kuhnian statement I’m giving you a state of the cut the current mainstream neuroscience paradise yes yes so I would and you know take this with a grain of salt because I’m super biased my argument is that this kind of functional mapping of the brain is what’s wrong with why you don’t understand and I’ll argue if you don’t mind specifically from neuroscience but also by analogy the kind of analogy I’d like to make is to another big mapping project the Human Genome Project which also when it was introduced people said geez this sounds a little unnecessary we already know where all the genes are why we have to map all the regions at orangy uh why do we care about that and it turned out of course that the comprehensive map revealed that the non-coding regions what we what I learned in college was called junk DNA is hardly anything but that was the kuhnian moment perhaps for genomes once we did the whole genome we realize things that we didn’t before that’s the first the second is if you think about how the genome worked nobody ever mapped the functional gene that’s because the function of something and the way some neuroscientists tested it’s just an ad hoc made up explanation for what a human brain thinks the function of a neuron is and we really gone really far with this there are visual systems there are pair of visual systems you know all of these things but human beings from you know ablation studies from a functional recordings I sort of set up these maps i if you made the functional genome and you said well it’s not a function of every gene it will be a complete again curse where do we complete as show the reason is that every scientist works on a particular gene thinks it has some weird function related to whatever it is that their personal history of working with that gene one and number two for Neuroscience the really crazy part is when we test stimuli on animals we’re testing like I don’t know less than 0.1% of the universe of stimuli we never looked at stimuli in combination you know very rarely audio visual olfactory etc and the response properties are super simple compared to the wiring diagram are the 10,000 connections that neurons have there’s something wrong about this there’s something along with sampling a few neurons with a paucity of a stimulus space across many different conditions who knows whether their eyelids are dilated what their personal history was just before these functional maps were taken and then trying to collate accumulate all of that it makes sense is as crazy I think in my mind as making the function achievement and he had a comical genome the anatomical Matt what I like is there’s an easy start there’s an easy finish in the genome I started with the first base pair and I got to end with the last base pair and say I’m done here’s the data set I give you the same thing in an anatomical map of the brain I can start with the first connection and with the last connection and say I’m done in these functional worlds I feel like it never ends at least I haven’t seen it converge okay was that controversial enough I’m really trying huh um that’s pretty controversial I think you know the many ways to kind of respond to that but it says look you know I mean and I’d like to begin with one argument which is that again if you’re looking at a global theory right I think your argument may hold water but we’ve learned an enormous amount from neuroscience right by folks I’m very small with a molecular circuit phenomena I mean you know all the lists of people won Nobel prizes you know often they worked out some particular system at least some of the mechanism some particular system so take the lab but I was in Richard Axel right um there you know they they work on smell and part of the idea has been first of all you know they they first cloned the genes in Mouse and then they went to flies which is actually sort unusual direction going to complex to simpler but they now be big and understand how the olfactory system works they can people critic or from hundreds of cells right now it’s a fairly small brain and we do have an understanding now of how smells are processed in the olfactory system of these sympathies animals so you know again we’re not aiming at a global theory but nevertheless if you view neuroscience is simply another version biology where a small theories of what dominate then the approach seems to have been bearing fruit whereas this sort of whole Anatomy mapping hasn’t really borne fruit so far as I can tell all right at a global scale it seems like we’re trying to solve a problem which we in our we I thought we kind of greed may not be solvable which is the brain in a kind of macro level so that’s that’s one of my my first response is now I think it sums as the analogy I think between the functional genome and say functional neurons is that taking it has led it’s sort of different we have thrown recently that their cells Lars of our cells in the brain right glial cells what you thought since he had no function whatsoever yeah and now it’s clear they do so that’s been a kind of revolutionary development maybe not certainly not on par with realizing that non-coding DNA is a fact regulatory and controls many functions of the brain but we now know to say something controversy I would say anybody who was smart knew that that could not all be junk DNA the protein coding could not be the only thing that’s happening inside your genome anyway and maybe you could say the same thing of people who thought the glia right yeah serve no function of the brain but here’s a case where at least there was a consensus of a large part of the field about who ever had as a function and now we realize that I could actually do it function so here we found in case of of the brain some region of the brain which was thought be non-functional now and opens up a whole range of investigations we can do so I’ve said so I guess I made a couple claims body which is again I still think that the standard paradigm has worked extremely well on the understand you were getting at small local theories the best thing do is probably start with simple brains because those are ones if you want to have another stay on the global scale you a better shop in the human brain and there’s a case where we’ve actually have come to see this something which was virtually non functional use function okay just on that last piece only because I like arguing so I apologize sorry get take two minutes and give you okay because I’m an assistant topic that I just want to make a case for simple and complicated breaks because I’ve seen this run or I’ve seen this stick a few times and I want to tell you my version of that stick my version of that stick is the average lifespan of Drosophila is a couple of days maybe a week two weeks I don’t know the exact number but you tell me I think they can we don’t know the natural lifespan but they can live in the lab you know up to 60 days okay two months no problem the number of progeny born per reproductive cycle is over a dozen maybe hundreds etcetera if you look over evolutionary time they’ve had way more cycles of evolution that we have the average lifespan of a human okay you know absent whatever is decades the average number of progeny born per reproductive cycle is way less so evolutionary pressure has been working dramatically on the fly and in fact it’s a much more evolved organism than PR serving six generals numbers alone and a lot of neuro scientists trying to fool this bait-and-switch where they say well I’m gonna work on a simpler system like the fly and figure out mechanisms and etc but it’s actually a bait and switch these animals these invertebrate animals are way more complicated well they’re more they’re maybe have they may have got undergone more natural selection than we have but in terms of strata viscosity of their brains clearly less right you’re a specific example of this okay in the fly and in the worm which is another model system it’s not that individual neurons compute at least in the worm different parts of the neuron compute so a dendrite a piece of a neuron might compute something completely different than the axon of that girl and those two might never talk to each and that immediately makes the system way more sophisticated at some level are complicated than these integrate and fiery the way we think about the entire neuron fire in the mail okay so like just let me let me reformulate that little so you’re saying that from a naive analysis it seems like the Drosophila or whatever it is is actually less complicated than say our brain however because it’s undergone so much evolution it may be processing information in a much in a radically different and actually more complex way even though it has sort of fewer atoms in it yeah in fact one version of this but you guys please push back is that a lot of these simpler organisms or analog computers they don’t actually work in the digital a large start large fractions of their brains are analog computer they don’t fire what we call an electrical spike when they reach some threshold and I don’t know the math you guys will know it better than I do but a lot computing in certain ways is more sophisticated than digital computing or complicated or etc so I’m not sure if the inference the understanding is going to flow from invertebrates to us I think you’re right you’re right to say that even if computer one has n1 components in computer 2 as n 2 and N one is much larger than n2 it doesn’t necessarily mean that the functioning of computer one is is more complex than number 2 because of the reason you gave that the actual met the actual mechanism by which it’s computing could be quite different and we have redundancy apparently humans have redundancy in ways that the fly does you can remove you know a substantial part of the humans brain we either by injury or etc and a lot of there a lot of the things that they do don’t get affected I don’t know if the same is true in this world of ablation in a fly brain where things are super specialized and perhaps less we’re done I’m gonna I’m gonna cut it there unless Cori has something really urgent he wants to say how could you not Cori I’m trying my hardest I don’t I’m not saying you were right but there there is a formulation of what you said which isn’t obviously wrong to me yeah look I think I don’t even delve too far into this read the code there’s no doubt that you know invertible neurons have major functions are often not unidirectional in the same way that vertebrate neurons are and they do they often do so to perform different functions the way the doors are actually much more specialized and something to be the lugs you’re having so many neurons we get up neurons which are relatively specialist beyond all the pups not a specialized neuroscientist how they pathi size and thinking you know that this neuron performs this particular okay function I’m gonna cut you because we’re running out of time sorry back to my one of the topic sentences that I mentioned earlier scientists are underpaid and society under invests in scientific research and I want you to react to that Bobby first and I’ll just um I’ll just mention that you and I have a common friend maybe a guy you went to Princeton with if you were Rhodes Scholars together and this guy is running a monster hedge fund and killing it and every time you talk to him on the phone you feel like killing yourself am I am i exaggerating perfect so tell us react to my sentence scientists are underpaid in society under vests in scientist under invest in scientific research okay I’m gonna disagree I’m gonna be very specific okay I think I’m underpaid good good I happen to be a scientist yeah I don’t know if that that it’s exactly into your motifs but a net total amount of payment not payment per dollar not payment per scientist it’s actually we have way too many scientists doing neuroscience right now so overall I think wait to my relative to what Bobby sorry relative to what scale return on investment what we’re paying and what we’re discovering how would you know what’s appropriate to I mean help you run this experiment right we have another sort of control set where things being run properly and we’re discovering lots and lots of things the same number of scientists right I mean how would you defend that claim right we have one experiment which is people doing science right now in the current world how many know how much you should be discovering one it’s a really good point Cory thank you for pushing me back on this I would argue that one way to analyze this is the more money and people you throw in a problem is the answer converging or is the answer diverge the more effort I throw at something am I arriving at F equals MA like am i converging on this idea that I’m gonna arrive at F equals MA or the more people that I throw at it the more equations actually come out that the results are actually diverging we’re not getting towards a unified theory of the brain we’re actually getting two more views but remember our previous discussion limits time suggests there is no unified theory in the brain right there are many many tiny tiny theories a small part of the brain and small phenomena associative brain there is no there’s no specialty there’s no general relativity there’s no you know fundamental laws of particle physics for the brain it’s kind of sad Corey what wait let me push back if you don’t mind it’s just physics in chemistry it’s not magical and we have taken complicated systems like hurricanes or and and and floor them understood them I see Steve say no already so you’re not also much simpler too many ways I said keep keep talking but how would ya simpler is this word I just never understand neuroscientists use it all the time etc but there will be an equation that a global equation I don’t know how many parts and how nerdy I have to be to understand it but it has to be right now one version it doesn’t have to be let me just push back on myself but it will totally screw all of us it’s a thought which is that the reason I can’t get enough I can’t analogize from physics or chemistry to biology because that biology has evolution thrown into it that’s the key that says everything that’s everything in this day it’s a haphazard system going together to keep organisms alive and reproduce over time no reason to think that me systematic laws really it’s happened there something gets there and it happened to work it stays yes and and that could be actually and I suspect that is actually what we’ll find is that there are very few principles when we do these kinds of global brain analyses and each little part of the brain is gonna have its own little rules out a wire up how to imagine etc I can buy that but at some point they’re all connected to each other meaning let me say it another way every neuron in the brain has to be connected given a certain number of connections to every other neuron in a brain has to be otherwise you have two brains two independent I’m gonna make that argument I suspect it’s true but I have no idea so on the nature of biology and evolution I don’t think we’re disagreeing actually we’re right she probably almost all and exactly percent I but I want to come back to the poignant plight of mid-career scientists like yourself are you not gonna are you gonna are you gonna lie I just got started really late okay way scientists said you’re that the your current point of career John yeah I think the listeners and the viewers want to hear something about what your day-to-day life is like the emotional rollercoaster why science is hard are you happy you went into it are you happy where you are in life most unhappy about everything that’s my general view etc but if you want me to drill down on specifically why I’m unhappy about science it turns out that at least in biology at least in my life in my burger shake once you get your own lab you never actually do experiments you never actually touch excite Nevers is a rough work it’s a very very rare to to actually be the person who discovers something they actually be the person who solves your management others yeah and in fact my life is the opposite I’m somewhere between an accountant and a salesman and an HR rep and that is what being a scientist is on a day to day basis I have to worry constantly about money to keep the lab going I often have to worry about relationships in the lab that are preventing us from getting data I have to think about who my who I should put as reviewers for my paper who I should exclude as reviewers from my paper was way more like human level thinking and relationships and I hated I’m terrible that’s why I didn’t want to be a doctor yeah I I do you think what his do you think his comments are comprehensible to the modal listener do we should we elaborate a little bit on the things he just said and they may help to elaborate so think as a picture of science which is yeah that’s more isolated yeah so much more I think the average person thinks a scientist goes in the lab it is thinking deep thoughts and maybe tinkers with their device and has a Eureka moment and everyone immediately realizes it’s a big discovery and they shower him with laurels and etc when in fact every day is a huge struggle you’re managing a team the team is really making the discovery if you’re successful enough to have people working for you yes well I’m assuming someone’s you know succeeding in the scientific enterprise you’re begging funding agencies and foundations to give you money to continue your projects you’re worried that you’re gonna have to fire a postdoc or a grad student if a particular grant doesn’t come through when you finally get your beautiful results your churlish peers fail to understand why it’s important they misunderstand you’re clearly written paper they reject it for the wrong reasons everyone is out for themselves did I leave any bad aspects out no I’m surprised yesterday we had a meeting with all of the assistant professors in the biology department meeting with the guy who decides our tenure and yeah it was an hour-long meeting where he said you know he wanted to teach us how to get tenure successful University of Chicago has about a 70% tenure pretty biology and I sat down he started off with you got a published papers you got to get grants and you got to be a good colleague sorry you have to teach excuse me publish papers get grants teach right and let me said that’s what everybody tells me but no oh that has very little to do with getting tenure yes a little bit but not as much as you think what really matters for getting tenure is basically the advertising campaign that you engage some mark along the way and he was right you know much reasons why you wanted to do this as soon as you get a paper out I don’t care anymore dude I already in fact I stopped caring before we publish the paper because I already knew the result and I’m interested in the next thing but that’s not how it works like you gonna publish that paper and then you’re gonna you know what do they call it a book tour I don’t wanna be out it’s so it’s funny that you say that Bobby because yes I mean the way people market themselves the way they push their results I think has a huge impact at least if they sort of elite level in science whether you’re gonna make become like a kind of superstar at Michigan State I it’s funny right before we recorded this I recorded a video message about the promotion tenure system here because I’m actually on the committee that that does that here and and how is try to emphasize how things are fair and you’ll be judged on your publication record and your grants and and I didn’t know where in my presentation in my video that a little video that I recorded it I say anything about self-promotion but realistically realistically we know it’s true and apparently at Chicago it’s it’s very important yeah we’re out of time so but I want to give you a chance to do one more thing it’s a last item on my list I heard you once say that professors must profess yes and I like to say professors love to profess yeah so I want to give you a chance to react or to answer the following questions where will neuroscience be in 10 years yes and in 50 years yes what will drive this progress yes what are the most exciting milestones that you anticipate and how should resources be reallocated if they should be reallocated within the field and I would love to hear Corey’s answers to these questions as well but not today and we have plenty of discuss sorry quarter there yeah so you get the last word okay thanks that’s like some Adam Sandler movie where he shows up for his final exam and they’re like we only have one question with 18 parts or something like yeah there’s only like six whatever fine you can you can leave anything out you want to leave out yes please in by that quote the professor’s have the professor uses I’m just sick of professors who don’t have opinions about things it’s pointless to not evident to have an opinion wanna attack opinions you wanna argue so I’m gonna use that vein I think in 10 years or 50 years there’s gonna be some we there’s gonna be a very little difference between neuroscience and computer science I’m not sure which is going to be a subfield of the other on my prediction is that neuroscience is gonna be a subfield of computer science the reason I think that I’m probably wrong the reason I think that is that right now you can’t really function a high at some high capacity neuroscience lab without having a set of your team or your collaborators that are essentially computer scientists to handle the thousands of neurons that were imaging in real time to handle the billions of connections the MA to track behavior or at keep track of terabytes theater computer scientists are already invaluable for their Oh science I suspected some point in the future I hope neuroscientist are going to become valuable to computer scientists because there are things about how the brain computes that’s going to be very hard for our hardware to do the one that I always talk about is that brains are twenty watts hardware is not when you’re talking about an algorithm beating go an algorithm leading a human it go it doesn’t beat that human when you divide it by the energy it took there’s a denominator the new the numerators how well it does the denominator is how many watts it took to achieve that result when you do that achievement over wattage human brains still kill and I think someday the computer scientists or the people who build out a hardware or software are gonna be learning from us hopefully just like they did with the very first machine learning algorithms like you were saying Steve the very first was she learning out where those it’s a very famous Japanese machine learning scientists who based the very first visual algal machine learning algorithms on the cat visual system on the network on the nodes of the cap digital system precedent is there I suspect the more we learn about brains the more they’re gonna find it useful and there’s going to be some mind meld between computer science and narrow side of course I never thought about it but it seems so obvious now everyday and we both been studying the same thing which is intelligence or intelligence but which is how brains get jobs done and I think that’s what will happen so that was two of the four questions I guess exciting milestones along the way I think that really for me the milestones will be reverse-engineering milestones will be where the things that brains do we figure out ways to put into algorithms and robots have a list well like you know it’ll start with maybe perception and move upwards or it might start with movement or motion move I suspect the wiring diagram of a spinal cord could help somebody make a better robot based on that kind of whatever control theory so I think those are the milestones that we’ll see in terms of funding you should really fund me and nobody how can we argue with that all right well unfortunately we’re out of time thanks a lot for joining us on the show Cory do I have any final this is been great Bobby your fabulous night at 22 I hope we get to talk more about neuroscience course I do absolutely and look I think this is a model we can get guests who are is fun engaging is Bobby I think we’ll good pretty well hope you’ll come back Bobby did I do accents and since you can’t see me I can come back as somebody else okay great okay Steve can you carry a pass the Turing test as Steve Schuh now I have a pretty good cockney accent alright well we’ll invite you back for the comedy hour okay okay bye-bye yeah well let’s stop there and if you enjoyed our podcast let us know can send an email to Corey or myself and we’ll see you next time