0:01 One of the great things about floresence microscopy is that of course fessence is available in multiple colors. So this means you can take component A of a cell label this in green component B label it in red component C label this in blue and then you know when you see blue you're seeing that third component. But there's a problem in that we're using molecules for fluoresence or atoms. And so this means that the spectrum of that emitted light is quite broad. And so if we try and label five things in a cell, we can pick five different pressin proteins which I've plotted here and we're going all the way from the blue end of the spectrum right to the very furthest red that you can get. And you can see that there's quite a lot of overlap between these different emission profiles. And so if we try to pick five individual bandp pass filters to filter out each of those individual colors that works for the first bandp pass filter
0:46 where you see that nice blue emission but then in all the other um filters you get a combination of two or more paroors at the same time. And so now there's no longer this onetoone mapping between labeled component and what you're seeing. So typically in fence micos experiments people have three maybe four colors and then that's it. They don't want to go any further because of this spectral B3 problem. But if we think back to the math that we learned in school, at some point we learned about simultaneous equations. And we can imagine that we have two detectors in this case just to keep it all nice and simple where on detector one that measures three parts of component A for every one part of component B and detector two measures two parts of um sorry two parts of component A for every two parts of component B. So [snorts] then if we measure 35 counts on detector
1:31 one and 30 counts on detector two by inspection you can work out that there must be 10 lots of A and five lots of B. If you want to make this a bit more formal then we can put this into a matrix form. And now we have what I'm going to call the mixing matrix throughout this talk. Uh that tells you how each of these components go together. And then in order to find the underlying uh quantities you just invert that matrix and apply it to your measure count. So you'll get back a vector of 10 five. This is not a new idea. You've been able to go into town to the microscope shop and buy a microscope that will do this for about 20 years now. This is one example. This is a Zeiss quazar system where you have a con focal laser scanning microscope on the left hand side and then a diffraction graating that takes all the light that's collected from that illuminated point, splits it up into a nice rainbow, and
2:18 then the rainbow is incident onto 32 detectors space one next to one another. So the detector on the rightmost side here sees the bluish light and the 32nd detector on the left hand side sees the reddest light. And so now we've got 32 different spectral bins. So you can think of that as having like 32 band pass filters, but rather than filtering out the light, they're just collecting that part of the spectrum. So now we can go back to our problem of having lots of flowores and we can make it a little bit more challenging and let's say we want to have eight flow of fours in our sample. We take our 32 different channels and we split up the spectrum equally between them. And apart from the very bluest detectors and the very reddest detectors, there's a roughly equal spread of light between all the different detectors. So we're not wasting any of them. We can then write down a much larger mixing matrix, which
3:04 you wouldn't want to handle by hand, but computers are very good at handling matrices. So you can just plug this into the computer and let it do all the processing for you. As well as being very large, you might notice that a lot of these numbers are very small. And that's reflecting the fact that because you've chopped up the spectrum so much, there's very little light going into any one detector. And this is a problem because that means you've got very little signal, which means your signal to noise ratio is very low. And so you can see particularly for these components down here, you get one photon in a thousand. And so that's going to be an extremely noisy signal that's going to corrupt as a later processing. And there's not really any reason why you need 32 different channels. If you want eight flow of fallors, then linear algebra suggests that you only need
3:50 eight channels. You've still got a perfectly well determined system as long as you've picked your bandp pass filters and flu fallors appropriately. So we could imagine instead splitting up the spectrum into eight bins which are now much wider and therefore collect much more light. And again, we've got a roughly even spread of light over all the detectors. Now we have a much smaller but still fairly large mixing matrix which is now 8 by 8. And now you'll see that all the numbers in these bins are significantly larger. So we're getting much more signal into each one of these bins. And for those of you who are fond of math, the condition number up there at the top is 12, which is not perfect, but it's all pretty good. It's not really any worse than what we had before. We had the 32x8 channel matrix. And that means that we can do this matrix inversion process process the data and get back back a reasonable
4:35 result. The nice thing about having eight channels is that before we had to have a single point that was scanned and the light split on the 32 detectors. Then we move to the next point and the next point and the next point. And so this is quite slow and it also means you got to put quite a lot of laser power into one focus point. With eight channels, you could imagine that you would just about be able to get eight cameras running together and detecting all that signal from the entire field of view in one go. And then if you couple this to a fast microscope like a spinning discon focal, this would then give you a video rate multisspectral microscope rather than a slow one that takes multiple minutes to acquire one picture. So this seemed like something that was interesting to investigate as a person interested in technology development. And then pops along Manu delivery who says that if we
5:22 built such a system, this would be great for studying end table sorting. So we teamed up and decided that we try and get a joint PhD student to take on this project. And we're very lucky and Akashi decided to join us and he together with Alex from our mechanical workshop built this instrument. And so what we have here is this is where your camera would normally screw into your microscope whether it's a spinning disc or turf or whatever you want. All the light goes through here through this focusing lens. And then this first dicoric mirror splits the light into bluish light going this way and reddish light going this way. Then we have a treel like arrangement of further direct messages that split up the light even further still. So we end up with the various light which you can't actually see on this camera sensor because it's not sensitively far infrared landing here and the bluish light being directed to
6:08 this camera. And so now we've split up this whole spectrum into these eight different channels. And because we're using doric fil sorry doric mirrors rather than band pass filters we don't lose any light. Every photon that comes in here ends up on some camera somewhere. You just need to work out where that is. So now we can acquire the data at the rate of the parent instrument. So if you were going at 10 frames a second, you can go 10 frames a second. Now you just got to worry about eight cameras rather than one. However, now we hit a new problem which is that if you use the traditional software for doing this spectral mixing process, it's a bit rubbish. And the reason for that is that the method I explained at the beginning pretends that there's absolutely no noise. And of course in live cell imaging, there's quite a lot of noise because you're trying to keep the laser power low and the exposure time short. So you don't memory the sample and you
6:55 also see all the fast dynamics. And so here we've taken a data set where we've got six components labeled and we've applied the standard linear mixing procedure to that data. And you can see that these components of the cell do not look like they're supposed to. If we zoom in a bit might make it a bit clearer. You can see that the supposedly nuclear channel has also got quite a lot of plasma membrane mixed in there. And the goldie channel is now Goldie plus mitochondria. So we've not got the right reassignment of the photons to the right components and so effectively this is useless. After spending a bit of time working on a better algorithm, we now get much cleaner signals back where now each component is correctly reconstructed from the mixed data. I don't really have time to explain exactly how that works,
7:41 but you can see in this movie that we start off with a bad guess of what each individual component is. In this case, the letters of the word spectrum. And as [clears throat and snorts] we iterate through this iterative algorithm, reassign the signals back to the correct locations. And so we end up with clean letters in our simulation. So now we can take this camera system, take off the camera off a swing microscope, port this on instead, measure some pointless functions to show that we haven't ruined the spatial resolution. And now we've got our video rate multisspectral acquisition system. So you label a cell with seven things. So here we've got uh the actin nucleus the endasmic reticulum mitochondria gioomes as the plasma membrane and then you just acquire images as normal do the algorithm and you get back your
8:26 individual components. However, now we have a new problem which is that if we try and do 3D imaging on a spinning disc, that works quite nicely. But if we try and do a time-lapse in 3D, then we see there's quite a lot of photo peaching and these cells are starting to round up and are generally unhappy. And that's not unexpected because speed discom focals are relatively damaging because you're illuminating the whole cell at any one moment in time. And so really what we want is a gentler microscope. [clears throat] Fortunately, we have one of those. Uh this is a so-called oblique plane microscope. Um but for practical um considerations, imagine it's an inverted microscope that also happens to be a light sheet. So you put your sample here in a normal imaging dish, a light sheet comes out at an angle. There's some horrific optics that then fixes
9:12 that and you get a nice image of that filtered plane on your camera. The other nice thing here is that the sample never moves. In order to do your Zstack, you just move a galvanometric mirror here and that moves the light sheet and the focal plane through the sample at the same time. So this is the only moving part and because it's a tiny tiny mirror you can move that very very fast. So we have once done 400 volumes a second. I don't recommend doing that. You don't see very much but we're not limited by the hardware in terms of imaging speed. So now we put the camera tree where the camera would go on that instrument measure the point functions again and they're as good as they were before we put the system on. So we've not compromised the resolution there. If you want to image really fast dynamics, then sometimes taking a full Zstack isn't really worth it because you don't want to see every individual plane. You just want to get a general idea of what the
9:59 cell as a whole is doing. And so with another galvanometric mirror we've put carefully before the chemistry system, we can do that using some optical trickery. And I'll just show you the end result, which is that now we're seeing the whole cell. So not an individual slice of the cell but the whole volume of the cell but still with optical sectioning because we've moved this light sheet through it and now we can acquire say 10 volumes per second but we only get 10 images per second. So the data throughput isn't quite as um ownorous to deal with and you still get a nice view of what's going on. So you can see here the mitochondria are going about their business fishing and fusing the GG looks a bit weird. Put more goss over that. Uh the endophas reticulum is wiggling around as it does. These proxyomemes are diffusing under brownie
10:45 in motion and then if you merge all the colors together you get an idea of what the whole cell as a hold is doing. At this point people normally interject and say yeah but labeling lots of things is hard. It's not quite as bad if you're prepared to put some effort into it. So if you're interested in organels or say the cytokeleton, you can now buy quite a lot of different live cell imaging dies from suppliers such as thermopisher and um spychrome add those to the cell media wash it out and for general imaging they seem to work pretty well and structurally they're nicely separated because you can get them in lots of different colors. However, if you want to actually look at like individual proteins or something within a cell, then you've got to make a plyronic plasmid and manu are very fond of this MXS system where by a careful choice of restriction enzymes, you can chain together multiple separate plasmids to
11:30 make one massive mega plasmid. And this is the system that we've used to make these six, seven, and eight color plasmids in the lab. I'm a physicist and therefore I have been trained to be lazy and so I prefer using Gibson assembly to try and get three or four things in one go. The trick here is we have some carefully designed barcodes and a synthetic promoter that avoids some of the repeat sequences that would normally screw up the synthesis when you order it from an old supply like Twist. In practice, this works some of the time. So, if you want to get it working properly, do this and spend a few weeks. If it's just a one-off experiment and you're not that fussed about it, maybe give this a try first and then fall back to this system. As an alternative, you could make friend with David Baker and get him to design
12:15 some proteins for you. So, [snorts] in this case, uh, a team of people working in David's lab designed, uh, proteinbinding proteins, uh, to different cells surface receptors. So, here we've got one that binds to alpha 5 beta 1. And you can kind of think about these as nanobodies, but rather than coming from an alpaca, it comes from a computer that has been designed to bind in a nice way. In particular, these have been designed to be pH insensitive so that when they get internalized and move through the endodmal pathway, any change in pH doesn't cause these binders to unbind. [snorts] And furthermore, they're designed so they don't contain any cyines apart from one right at the end, which is where we attach a single floral. So we don't have to worry about working out how many copies of the floral there are like you would do if
13:01 you use a normal happy dot. So David designed these. Carrie and you spent a lot of time purifying them and labeling them for these experiments. And in terms of what it looks like, it looks like this. So you incubate your cell with these different binders. Leave it for a while. Wait for them to start internalizing. On the right here we see all the colors overlaid. But on the left, we have what we call a pair-wise interaction montage, which basically means you've got one thing in blue going along the row and another thing in yellow going down the column. And so you're seeing a twocolor merge of those two pair-wise interactions. If I play the movie, it actually doesn't look so good on the projector as it does on the screen, but you can see that there are different endomemes of different colors moving around. And if you look very carefully,
13:46 you might see that some of these endomemes change color over time. And to zoom in on that, if you take a single still frame, we can do the in mixing and then get back to our individual receptors rather than the mixed signals we actually record. And we can see that different endoomes, so these colored boxes here contain different receptors and different lials. Furthermore, because this is a movie, we can follow a certain endoome over time and see what happens to its cargo. So for example, here we've zoomed in on one endome here, which then fisions into two. And you can see the transferring receptor gets roughly equally partitioned into those two daughter endomes. Whereas you can see that the EGF doesn't go into the sort of bottommost endome. And so with this now we can study how these different cargos are sent through different pathways and
14:34 for the first time to our knowledge we've actually put a time scale on this because we've seen that this happens within two seconds. So if we try to see this on a normal spinning disc microscope where you're taking plane by plane imaging, you just wouldn't be able to see this in 3D in the whole cell in one go. And then because it's a movie, we can also track the particles to find out where these different cargo go. At the moment, this is done semi-manually and we're very interested in automating this all. This is proving to be extremely difficult. So if anybody has any suggestions, please let me know. Um, but the end goal is to be able to follow every cargo that gets internalized, see where it ends up, and see how these mix within the cell. So the problem with doing methods development is that you never really know when you should stop and when you should move on to something else. But
15:19 fortunately, we had a conversation with Sher here who worked out something called the Krameral lower bound for the indexing problem. And basically what this is telling you is that mathematically irrespective of what algorithm you use, you can't get an accuracy that's better than given by this equation. In our testing, our algorithm gives us an accuracy that's equal to what this equation says it should be. So there is no algorithm that will give you a more accurate result than the one that we've developed. So we don't need to worry about making it more accurate. It would be nice if we sped it up. So we sped it up 4,000 times. And so now rather than taking a second to process a single image, it takes less than a millisecond. So in theory you could unmix in real time never actually see the raw data and just be seeing your separated components as you require it.
16:04 In terms of applications a cash is now finished his PhD and has moved across the road to simmit Jillian Griffiths using um the multisspectral system to look at how organels are polarized during T- cell killing. Megan who's somewhere in the audience here uh is partly working with Alex Johnson from exit to look at endoccytosis but this time not in malian cells but in plants and then Luca and Zisha from the clinical school in Cambridge are interested in seeing how mitochondria interact with different organels within cells both during inflammation in mitocle and after their novel drug treatment they've invented. We've also gone a bit multisspectral mad and so now our plan is to put a multisspectral imaging system on every instrument that we develop from now on. And so eventually we'll be putting one on the cryofresence system that Jacob's
16:50 been working on with Chris Russo and we'll add one to the imaging flowy that Bill's been building with Pier Andre. So now we move on to the next problem which is death. Now death will come for us all and this is true for your samples as well. And in this case, I've tortured this poor cell by blasting it with blue light while acquiring my movie. Eventually, it kind of gives up. But it just fights on for a long time until eventually bop. [laughter] I'm sorry, but it had to be done. [snorts] So, what's going on here? Well, we need to revise a bit how the molecular mechanism of frescence works. So here we have a Joni diagram or energy level diagram where you have your electron sat nice and happily in its ground state going about its business.
17:36 We'll flash in some blue light. This will excite it up to this upper singlet state. After a bit of a delay, it will then emit green light and decay back down. This is the process of fluoresence. This is what we want our molecule to do. Sometimes we hit it with blue light and the electron acts back down and doesn't give out a photon. This is internal [clears throat] conversion. This isn't the end of the world, but it would have been nice if we got that photon that threatened forgiveness. The worst possible thing is something called intercession crossing where we hit it with blue light and then rather than relaxing back down to this ground singlet state, it ends up in a triplet state. Now, it turns out that for GFP, this happens roughly 1% of the time. So, you're actually making quite a lot of triplets. This is bad because once you're in a triplet state, it's quite hard to get out of it again. And so now
18:22 you have this molecule that's got this extra little bit of energy wandering around looking for oxygen to react with. So option one is it just inter crosses back. Nothing very particularly exciting happens. The electron is now relaxed again. Option two is we now get phosphoresence which is at a lower energy and therefore a longer wavelength and so you get a little bit of light out but again your electron is relaxed. The third possibility which we don't want is that it reacts with something particularly molecular oxygen because that's also in a triplet state and then we get photo breaching and generic destruction. So we would like a method which I've called t to try and get electrons that are in the triplet state back down to the singlet state. And so the original
19:08 idea was to just blast it with red light corresponding to the phosphorescent transition because then you should be able to drive that electron down by simulated emission. So you put a lot of red light on and eventually we coersse the electron to fall back down without reacting with anything. In order to test this out, we want to use a die that produces a lot of triplets. And we know that halogenilated dies such as B are very good at making triplets and also have large indices crossing rates. So we should be able to see a good signal there. This is also a nice uh die to pick because you can actually see the phosphoresence. Normally phosphoresence is so weak that it's quite hard to detect, but this makes loads of it. So we know exactly what the spectral character is. So the plan is we excite normally with 532 nm
19:53 light of sort of thin film of the die and we get some signal. We then excite with 660 nmter light which corresponds to that phosphorescent transition and we don't want to see any signal because that means that we're exciting from the ground singlet state to this singlet state. We don't want that. So we see no signal and then we put both on together and we see that we get much more signal and so this seems to be working. This 660 light is now pumping triplets back to this emissive singlet manifold. And if you quantify how much more signal we get, we get about four-fold more. The problem is if you go away and read a few laser physics textbooks and learn how to do some calculations about what the sort of scalless effect should be, you find that that's way too much for
20:38 the laser powers that we're using, you shouldn't get that kind of enhancement in signal. And so this idea that it's stimulated emission that's driving the electron back down to the ground state is probably wrong. So what else could it be? Well, it could be that there's a now a higher level triplet energy level that we can pump into. And now when we apply the red light, it goes up to this triplet level and then there's a reverse indecision crossing back down to the singlet manifold before falling back down to the ground state. And this is kind of supported by this idea that it saturates with 660 power fairly early on. So we've made the molecule four times better than it was before. But is that because we picked this particularly weird erthin B? Can we make it work for other dyes or even better frescent proteins because florescent proteins are the ones that bleach and damage things
21:24 the most? Well, there's some hint that we can because for the past 15 or so years, Robert Dixon's lab has been working on what they call optically activated delayed floresence where you do this pump with blue light, hit it with infrared light and magically get back some fence photons later on. >> [snorts] >> And more recently there's been a very very detailed study about the triplet state within green fluescent protein and they found there that this triplet state has a lifetime of 5 milliseconds which is ridiculously long in comparison with a normal die. You might be of the order of 10 microsconds. So when you get into this triplet state which remember happens with 1% yield this electron is staying there for a very long time and therefore has plenty of time to bump into something to react with. This uh Jack's paper also did some
22:09 transient absorption spectroscopy which is very difficult to do. And the idea here is you excite the molecule and then see what is absorbed after it's been excited. This corresponds to the normal ground state absorption. And this is the T1 to T2 or let's call it TN because we're not sure it's T2. The higher order triplet absorption. They also measure the phosphoresence which you can see lies between about 650 nmters to 850. So if we want to repeat this ereithin B experiment but now avoiding stimulated emission from phosphoresence we'll cross out that region of the spectrum and then pick a wavelength with good um excited absorption. If you pick here that's a 561 laser we've got those icon focals that's quite easy. If we do that we find that we actually increase the bleaching rate. So we've
22:56 made the GFP worse. Then the question is what happens if we go for this other peak though? Maybe maybe that would work. Turns out that actually a 950 nometer laser at high power is quite cheap. So he bought one and then the supplier forgot about her order for 6 months and so had to wait a long time which was a bit frustrating but it arrived. You pump in with 915 lights and you get a four-fold increase in the photo bleaching rate. Sorry I say increases and it's four-fold better. So the rate decreases by a fracture four. Then two days later this paper comes out to biotechnology showing exactly the same thing but in a much nicer way and for a much broader pallet of feric proteins. And so this seems to be a general trick you can use for green frescent proteins to reduce the photoing rate. Question then is what's going on?
23:43 And the answer is this. This is the set of differential equations that's governing the uh transition between the different energy levels. And I'm fairly good at maths but I can't solve that. So I have to put it into a computer and do some numerical um simulations instead. But if you set aside two weeks of time, lots of cups of tea and stare at these equations, eventually you'll come up with this equation which tells you this enhancement factor. So this epsilon here is telling you how much better your photo peaching rate is. So if this number is less than one, you've made things worse. Things peach faster. If this number is greater than one, congratulations. You now made your flow for last longer effectively for free. We don't know what any of these numbers are, which is a bit unfortunate, but we can kind of imagine the sort of plausible ranges that they might be in.
24:30 And if you take all those together, you kind of come to the conclusion that this epsilon can't really be any bigger than 10. [clears throat] And if you compare that with the experimental data from that paper I mentioned, that's kind of what they find. Even the best crescent protein, M clo Clover, gets a factor of nine enhancement. So it's possible that there is a phenant protein out there that works magically for this and does give you much better enhancement, but so far we think that a factor of 10 is the newest and unfortunately M Clover was really terrible to start off with. So we've made it nine times better, but it's still rubbish. GFP is still the winner. We've now made it four times better, but not infinitely better. There's also another issue which is that if you add this laser onto a sample that's got a red frescent protein in it that makes it photo beach faster which
25:16 for multisspectral imaging is not good because we always want to have green and red and blue and yellow. Commercial instruments don't have near infrared lasers. We have no idea what the spectral behavior of this process is although we're trying to work that out with some collaborators in Sheffield. And even in the best case that we know of EGFP we go from a million photons to four and a half million photons. If you compare that with a fairly standard small molecule die, they're in the sort of 10 to 100 million range. So what we should do instead is just use this super stable comparison protein stay gold which has 16.5 million uh photons within it. The problem is it's only available in water color which multiple imaging is a bit disappointing. And so now we're trying to shift that spectrum and also apply what we've learned about the photoics of stagold to
26:02 other frescent proteins to make them more stable. We've had some limited success with this. So, we've made stay gold into stay gold up because it's slightly more yellow than before. Um, and this is still fairly stable. We need to do a few more tricks to improve the folding, but fortunately this week Tom has joined us as a PhD student working with Buzz Bound to make crescent proteins great again. And so hopefully in a couple of [clears throat] years time he'll be able to tell you how that's gone. So with that the paper for the multisspectral imaging will come out in hmatics on Monday but in the meantime you can read it on bioarchchive here. Um [snorts] I should thank all the authors on that paper particularly a cash ker us and manu for their efforts in making that work which was not so straightforward. uh I've already introduced these people for the broad spectral applications and
26:48 what I haven't talked about in the interest of time is the exact details of the spectroscopy we've done to understand the photos of these quantum proteins but particularly the transient absorbent spectroscopy from Jenny Clark's group and the pump protocopy from Andy and Juliet have been very useful in understanding that and Vick's been very helpful in actually purifying these proteins for use here I should also thank our funders the Royal Society and the medical research council and you for your ATTENTION THANK DO WE HAVE ANY QUESTIONS? OKAY. >> What's the power we using for the and what's the extra damage that comes when you do that? >> Uh so for the best results for GFP, you
27:35 want about 2 kW per cime, which is pretty high. uh in terms of thermal effects because it's spread out over sort of 100 or 200 micron square the the thermal heating is negligible and as far as we can tell nothing absorbs that light directly so even though you're really blasting it nothing really responds apart from the GFP and the GFP responds in a positive fashion >> so you can go all the way to getting more signal >> yeah so it it kind of cons out around 2 kW per centimeter that gives you your 4.5fold enhancement and And you can turn the laser power up further and you don't get any more signal back. >> So changing the photo physics from stay gold to stay golden. >> Are you like are you editing the
28:20 molecular structure of the actual chromophore itself or are you changing the electronic environment slightly around the chromophore or a bit of both? >> So we've tried both. Changing the chromophore is a bad idea. That turned out to be really hard. Um, for this particular one, we've just put a pie stack above the chromophore and then tried to work out how to stabilize that so we don't lose the quantum yield. Um, for the other colors that we're trying to make, we're actually starting with other florescent proteins that already exists like mcar m those kind of things and then putting in an environment that kind of mimics that stay gold environment around the chromophor to see if that gives us the spin orbit coupling that we want. >> Yeah. Hey, really nice talk. So um
29:06 you show initially that you could do as eight colors and you had a fairly good condition on the matrix. What do you think the maximum number of colors ever you would be able to get? >> So to a first approximation that depends on how narrow you can make the emission spectra. So with normal florescent proteins probably 10 maybe 12 is the absolute limit if you're prepared to go a bit further into the infrared as well. If you're prepared to use quantum dots or some of these newer frescent proteins which have tighter emission spectra because their quantum yield is higher, we think you could push to maybe 16, maybe even 20 if you're prepared to go a bit into the UV with your quantum dots. But somewhere in the order of 10 to 20 is probably feasible. If you want to go higher than that, then you'd have to use some kind of barcoding strategy where you've labeled it with multiple colors,
29:52 but the number that you see also tells you some information about what you've labeled. >> Okay, thank you. Can we thank James again please?
