Counting Sand

Bonus: Season 2 Recap

Episode Summary

As we conclude season 2, Angelo reflects on the many brilliant guests and shared insights throughout the season. As Moore's Law ends, how do we continue to innovate and adapt to the ever-increasing domain of data and performance? What's next in computer science? Will the innovation and invention cycles that took place with Moore's Law repeat themselves in the golden age of quantum computing?

Episode Notes

In a time crunch? Check out the time stamps below:

[00:45] - Moore's Law, where do we go from here?

[03:00] - How do we improve data system efficiency?

[10:30] - Purpose-built systems (FPGAs)

[11:13] - Insights on (FPGAs)

[13:32] - Event Streaming

[17:50] - Data storage

[18:34] - Google’s approach to data storage

[19:00] - Downtime

[21:06] - Serves impact on environment and solutions to optimize

[23:00] - Improving data systems, machine learning, artificial intelligence

[24:06] - How do we regulate AI?

[26:10] - Benefits of simulations through machine learning

[28:38] - The impact computer science has on astrophysics

[31:09] - How do we defy Moore's Law, the future of quantum computing


Our Team:

Host: Angelo Kastroulis

Executive Producer: Náture Kastroulis

Producer: Albert Perrotta

Communications Strategist: Albert Perrotta

Audio Engineer: Ryan Thompson

Music: All Things Grow by Oliver Worth

Episode Transcription

Angelo: I want to start by saying how lucky I am to be a part of this podcast and to have really great listeners, like all of you.

I'm recording this final episode of the season from Crete, Greece in our studio here and really just pondering on the fact that I really get to do what I love to do. I love computer science, I love data systems, and I love having an opportunity to be able to share it with all of you. Either from our studios in the United States or here in Greece.

I'm your host Angelo Kastroulis and this is the Counting Sand podcast.

This season has really been about where do we go from here? Moore's Law has been a perfect vehicle to describe that. Moore’s Law states roughly that every 18 months, computing power will double. Now we know they can't go on forever because in episode one, we talked a bit about how transistors are physically packed onto a chip.

Angelo: Dr. Robert Caldwell, who used to work at Intel and was the chief architect behind the Intel Pentium chips between the Pentium Pro and the Pentium 4, later became director of the microservices technology office at DARPA. And he said back in 2020, that by 2022 Moore's Law will have ended because of the five nanometer process.

Well, why do you say that? Because we could still go a little further. Well, the three and the two nanometer process and who knows? Maybe a one, I don't know. Three and two nanometer processes have not yet been proven to show such a substantial improvement over five.

That was his argument. That three may only be marginally better than five. But you'll pay a bunch more for it. So then Moore's Law would have come to an end further before that. So it isn't necessarily tied to the wafer process.

Now it used to be that CPU was the bottleneck of computing systems. So a lot of our algorithms track that the algorithms of the day that started winning were the ones that were efficient with CPU. And the ones that dealt with data movement were kind of put on the back shelf. Then over time. It shifted to where the CPU, just through Moore's Law, kept improving so much so that they were outpacing data movement.

And then the movement of information from disk to memory and memory to cache were the big bottlenecks then of course, disks got faster and eventually you'd have so much RAM on a machine that it was just memory movement inside of RAM. That became the problem. So the problem has been kind of

moving, but eventually that will become so improved that we'll be back to CPU being the bottleneck and the fact that we have no more room for improvement, can't make it any smaller. Will make that bottleneck insurmountable. So what did we do?

And that's what we've talked about this season. What kinds of things can we do to have improvement despite the fact that we've been relying on computing power for so long.

I couldn't think of a better way to end season two, than to have Manos Athanassoulis back on to talk about his very relevant work in the topic we're talking about. What we ask of a data system is called a workload and there are different kinds of workloads. Listen to what he has to say about that.

Manos: People have been doing research on column versus rows, even from the eighties. However, it's still relevant because many of the workloads that we're executing have this dual nature, and now it's becoming more and more that this dual nature exists at the same time, all of the time. Right? So on the one hand, we're collecting lots of data measurements, information about sales, information about monitoring activity, whatever data we're collecting, we're collecting lots of them at any point of time. And the natural way of collecting data items is putting together all the conceptual pieces of the data item. And that's what they're always essentially. So a row can be for example, I want to store all the metadata that I captured from a group of sensors at this specific timestamp, or it could be all the metadata of monetary transaction that you made online. I want to show all these metadata, but then again, every now and then I might want to analyze my overall data collection to figure out some trends. So maybe I want to make sure I find, for example, how does the temperature change across seasons from a specific sensor or how my sales, how do they change as I am moving into different products or different locations and so on.

Angelo: What Manos is describing here is a way that we conceptually divide workloads. We think about them in different ways. He mentions a dual nature. One way we can think about them is kind of tearing through the data to categorize and group the information. And another way is to think about it in terms of keeping it intact with some of its context.

Why would you wanna keep context? That particular arrangement of information comes handy when you're trying to do certain kinds of workloads? What kinds of workload? It could be groups of sensor data or it could be your bank balance along with the transaction information for that particular atomic charge. We typically call those kinds of things rows. A row of a certain kind of information has these columns in it or little bits of information that we want to keep together.

An example of the other type of workload is where we want to do something like compute an average or compute a total. We aren't necessarily interested in each individual transaction, short of just getting its total while we compute. And then after that, we don't need it anymore, because we've aggregated all of this information. That type of workload would not benefit from keeping all the context. In fact, that kind of gets in the way, it's noise. I'd rather be able to arrange the data a little differently to just have all the totals together so that I can add them up.

And these two kinds of workloads are significant so we tag them with specific names.

Manos: So the first one is what you were calling transactional and the second one is what you were calling analytical. And they typically favor different ways of storing the data as you call them rowstores and columnstores accordingly. But also this means that there's a different architecture of the system. So to make a long story short, when we are collecting these measurements one at a time, essentially the easy thing to do is to simply append the whole measurement every time we get one of those, right? If we want to say, find the average temperature of a specific sensor, we are interested only in this temperature information. And we don't want to be reading, at the same time, more data when we’re asking this query.

Angelo: And in the past, we used to refer to these as OLAP, online analytic processing and OLTP, online transaction processing, to kind of denote the two kinds of workflows. But that terminology doesn't seem to describe systems adequately nowadays, at least more modern systems. Why? Well, Manos described a setup in which, what if we were able to take the two pieces kind of together?

What if a data system had aspects of both? We could maybe have, he called it fractured mirrors, where we can have one mirror copy being the data arranged in a columnar format looking at it from an analytic standpoint. Or we can have another copy that's more of a row specific format, which keeps the context of the row together and lets you treat it as a transaction. Now, it isn't just the way we arrange data, it also has to do with the capability to be able to insert and change data. Transaction systems have to have certain compliance. We've talked about this on previous episodes. The ACID properties, and that's an acronym, A-C-I-D.

What it means is that we make certain promises or guarantees. For example, the A is that the data is atomic. That it's a unit. That it's consistent. Meaning when we put the data in, we can turn around and read it and we're gonna get the value we just put in. We also have these ideas of transactions. For example, either all the data makes it in in a group or it doesn't make it in an all and we don't leave

the database in a half state. Those kinds of guarantees are part of that.

And remember there are certain trade-offs and benefits to the way that we store data. We can optimize it for reading, for example, and for writing, we've talked about that a lot. Others have certain structures that they are able to use to find the data very quickly and efficiently. So you have to always make trade-offs when you're making these kinds of design decisions.

This idea of fractured mirrors means we can make completely different trade offs. We can trade-off space. For example, maybe disk is cheap and be able to have the best of both worlds by keeping the data stored in two different ways.

And then we just have to decide which of these to go down. By the way, that was my thesis. How do you decide which of these paths to go down? Well, now you added more complexity and you created a more complex system, but now you're able to kind of expand its use to straddle these two things, OLAP and OLTP.

And what I'm trying to lead to here is that modern systems can do these kinds of things. They can tackle the problems differently, making terms like OLAP and OLTP not make as much sense anymore. Those are 20 year old terms, and that's why they don't adequately describe what we're doing, because we can now build data systems that are a blur between the two.

I'm going to give you an example on which most modern data systems are built. And that is the C in the ACID properties. The C stands for consistency. If we could relax that just a little bit. Consistency, remember, is, you give me a piece of information and if I turn around and ask you for its value, I'll get the right answer.

What I just gave you. Now what has to happen in order for that to be true? I have to make sure I pushed it through the entire system. And if this is a thousand node cluster, I will have to have pushed it through the whole node.

Every node of the cluster. That's a tremendously high latency problem. By the time I push it all out, you're going to be sitting there waiting.

I can relax it though and say, it's okay if I get the previous answer for just a little bit, until it gets propagated through the whole system. Now I've opened the door for extremely high write performance and many, many nodes working on the same problem. I know what you're thinking. There's no way we can make that a reality. We can't relax it. Yes, you can.

In fact, as it turns out in practice, most of the time you can. I've even seen papers

that make the case that it can be done in almost every instance. In other words, when we think we can't, we probably can.

What does this have to do with the end of Moore's Law? Well, if we can't have a system that can do everything well, we have to now build purpose-built systems that are good at certain things. And we have to start making systems a bit more creatively. We have to start making a system that can maybe even arrange itself, change the way that it stores data or understands the information based on what it is you're asking of it.

There's some interesting work that Manos was doing with FPGA’s. Now these are field programmable arrays that can be put right next to the CPU. What that does for you is it lets you rewrite the code for some of the functions. So we can actually put some data system functions directly on a chip later on in the field as a data system is maybe installed on this system. Listen to how it works.

Manos: Essentially you can implement whatever you want in this small FPGA. It turns out that I was discussing this idea with a colleague, BU, Renato, Renato Mancuso who was working on real-time systems. And he has a lot of expertise in building new hardware with FPGA. And, he was also working on an idea to use an FPGA which is located between main memory and CPU to actually make the CPU believe that a specific memory address is available or is not available.

Angelo: Okay. So we can't just rely on sheer computing power to be able to get us where we need to be because we're going to run out of that here soon.

So then you might be thinking, well then why don't we take many of these packages or many of these cores, chips, in different machines and use them at the same time.

And in fact, we do, that's called horizontal scaling. So what we'll do is we'll take many machines or we can call them nodes and we'll run the computation on many machines. And that is the modern way that we kind of have gone around the fact that we've got this limitation in compute. What we do is we try to divide the work up.

There are several problems inherent with that. For example, not all work can be divided up evenly. For example, let's just say you want to compute the average of some set of numbers. What would you do? Well, you would add up all the numbers and you divide it by the number that's in the set. And so if you had it on one node, one computer, one CPU can run through the entire list, total them up and then do one division. How do you do that on many nodes? Say that I have a thousand computers to calculate one average? One way to do that is to send an

operation out there where you're telling each node to just sum up some set of numbers.

You break up the set of numbers into a chunk for every one of the nodes. They each sum up their number and then you take that and you collect them all, the thousand, and then you sum up that thousand and then you do a division of a total. So now I can do that in parallel or actually what it really lets me accomplish is to sum up a number that no one node could do. This is called map reduce. This takes me to a whole other way of thinking about problems and that is using event streaming.

In episode five, I had a good friend of mine, Anna McDonald from Confluent, talk a little bit about some of the things that she's thought about event streaming. Now this is just thinking of the problem differently.

Looking at data as different. Rather than data existing in data systems we think about data being in movement. If it's constantly moving and we put it in logs and kind of process it as it's needed, we've changed the problem. So we're able to now rethink it in new and creative ways. And again, we're able to put this on many nodes and so we can process much more information than a single node could.

And in fact, my firm is asked all the time, can you help us modernize? This is what they mean—thinking of it in terms of an event architecture. The problem with something like this is it requires you to think differently. Just like we challenged this idea of a relaxing consistency, you have to think of the problem as data in constant movement.

And again, the same kind of rule applies in practice. It turns out you can think about many more problems than you think you could in event streaming if you just think of them completely differently. Listen to what Anna has to say.

Anna: The people that had the hardest, kind of mental model shift with it, were people who had been programming in Java and actually for us, where I came from, those were the people that got to do the new shiny things.

They had the biggest problem with a mental mind shift. They were like, no, wait, what? I'm like, there is no end. So to me when we talk about streaming, that sums it up in one word, there is no end. There is no certainty. This is not the state you're looking for, right? And you have to be comfortable with that when you code, when you develop, when you think about these things. It's continuous. You can't stop to catch your breath.

And I think there's a lot of people who have spent a lot of years doing object oriented development, where they're used to stopping and catching their breath. That doesn't work.

Angelo: There's some limit to how much I can put on one computer. So now I can sum up, or average, very, very large sets. Well, it'd be faster if it was on one node, but one node can't handle that. And actually the time we take to separate it all up and send it along to each machine and then collect it and then take the intermediate results and sum those and then do the division, all that time we waste doing that still is worth it because one node couldn't keep up. However, think about all that work and all that orchestration we have to do. That adds a lot of complexity, but still, if we had a thousand computers working on one problem, how much space would that take?

Let's think about that in terms of physical space. Let's say you have a laptop and the laptop is about 15 inches wide and you can close the lid and you can stack these laptops. Of course you need some space between them. And the laptop might take up only a small amount of your counter.

But a thousand of them need to be stacked and if you could do this perfectly as a cube with only a little bit of space between them, you'd fill a room and that doesn't even count how much supporting equipment we need.

These have to be wired up together in a network. And so you're going to need a thousand port switches and then you're going to need a power bank that can run a thousand laptops. And if a laptop takes about a hundred watts, then you have a thousand times a hundred watts of power.

Do you see what I'm getting at? It's actually a little bit more complex than just saying, well, let's just spread it out on a thousand nodes. And so now what you have to do is build a building that can handle a thousand nodes. But here's something else to think about. These computers are physical pieces of equipment, subject to the same kind of rigors and anything else the natural world is subject to.

So then let's think about it this way. Let's take a modest failure rate. Let's say that less than 1% of these machines will fail. That's of a thousand computers, that's 10 failed computers. Not to compound the problem, but we actually need space to be able to get around for a technician, to be able to get to the computers and unplug them and replace them.

The international data corporation estimates that stored data will need to increase by 17.8% by 2024. So to compound the problem further, we're not just dealing

with the failure rate of those pieces of equipment, but we also have to continue to grow the space because more and more data, their living machines, data is going to be put on those machines and someone has to get in there and replace the disks or to make them bigger because the data will change.

Do you see the problem we're getting at? If pieces of equipment are failing all the time, it's not just the fact that we just have to disperse the computation and that's the end of it. We now have to deal with all these other realities. In fact, Google and their data centers have shopping carts full of hard drives that they're constantly roaming down the aisles and replacing.

Google uses artificial intelligence to try to predict when the hard drives will fail so that they can have an idea of how many drives to buy and where they should be and how they're able to deal with this kind of problem. Why? Because they care about downtime. So let's think about that perspective for just a moment.

We haven't taken that into account at all in our computation. If, in our scenario, We need to maintain an agreement that the computation must always keep going because it'll constantly fail and restart when machines fail, especially more likely as you add more computers. So ten computers in a room going down is not a good scenario.

We can live with this, though, if we put in a bit of redundancy. So what do we do? Let's just say, okay, add 10% more computers so that we can account for that. So we'll have enough to cover that and then some spares. That gives us a little bit of wiggle room in case there's an outage. But is 10% the right number?

Should we have more? I mean, that's kind of a razor thin margin. So let's say I increase it to 50% and I need 1500 machines. Well, now I need a bigger room, maybe more people. I now have new single points of failure. I need bigger routers. I need bigger networks. I need more power. It's untenable.

That's one of the things I want you to get out of this season is that we can do interesting and different things. And we can take ideas that have been around a long time, reassemble them, and rethink the problem.

And to do that, we have to build complex systems like this that are able to do the kinds of things and still give us a service level agreements we expect. But there are more solutions in just thinking of the world in terms of columnstores and rowstores working on individual components of a data system and try to squeeze every little ounce we can. Before I take us there though, I want to take us on the little journey down this season to talk about some of the downsides of this idea. Like in our thousand computer analogy, we left it with, okay, now

how do we power all these machines?

In episode eight, I had a conversation with Mats and John and we talked about a different idea. Data centers now will have an energy problem because we have to produce more and more energy. And then these data centers produce heat and creates all kinds of other things we have to resolve.

What if we were able to push this problem to an edge device? In other words, before you even get to me, can you do some pre-processing and then I can distribute it a little bit more efficiently within my data center. We'll talk about that in the next season, a little bit more. That's edge computing. But listen to what John had to say.

Jon: I think the other aspect that we need to consider is, and this is where data centers are very complex to operate. So, servers have multiple bio settings and one of the things that happens in the industry is that a lot of the folks that install these servers see the word performance mode. They put the servers in performance mode. And of course, what performance mode does is it kind of disables a lot of the energy saving features, but makes them extremely responsive and react really fast to workload requests. And so you get the situation where you don't see so much variation in the power, because they're all drawing a lot of power all the time, because they're in some kind of state that is a state that they can be in, but it's probably not the ideal state for operating today.

Angelo: Techniques like what we've talked about are not the only ways to address this problem. There are more cutting edge methods like machine learning. In episode seven, we met Stratos Idreos, my thesis director and professor at Harvard university. Some of the work he's doing in the data systems lab at Harvard is using AI techniques to be able to improve data systems themselves.

But in our conversation, we stumbled across some really interesting points. For instance, machine learning doesn't always solve the problem. It's not always the best tool for the job. Sometimes you can just figure out the problem in a computational method. If you just knew where to go. Using machine learning as kind of a way to bloodhound your way to the right solution, using it as a research tool, is fascinating and can lead to better and quicker data systems.

Stratos: ML is an absolutely amazing tool to use in many occasions, but not when you know what to do. Many times in our research we use ML so that it can point our intuition in a direction that makes more sense than what we're looking at.

And oftentimes we find that, oh, because an ML algorithm is converging in this direction, we might actually be able to understand the problem better and then we'll move on and many times we are able to create an actual mathematical solution out of this direction. And oftentimes we are unable to do that.

And this is when we rely on ML. So we use ML very heavily in our lab, both to lead our research in directions that maybe we're not seeing the patterns and then we use it as inspiration. And then if we can find analytical solutions then we go with that because this is always preferable.

And if we can't, we rely on ML. You know, we have kids. It's very similar with what happens in real life. Right? So you see a kid the way they grew up, they try out things but once they know something, they don't repeat the error. Right?

So, when you see a one year old trying to stand up under the table, they hit the head three times. The fourth time, they know they have to move away. They won't stand up below the table. It just doesn't make sense anymore. So we try to use ML in the same way where there's exploration, there's learning but then once we know we don't have to learn anymore for a particular context.

Angelo: Machine learning, though, presents a whole new set of problems. For instance, how do you regulate machine learning algorithms? What do you do with a bias that's inherent in the algorithms? In other words, what they learn based on the limited set they have and we try to apply that to a bigger scope.

In episode nine, I had a great conversation with Eric Daimler about that. And some of the methods that the federal government in the United States is using to regulate that. And what do we do about it philosophically? Listen to what he had to say.

Eric: Yeah, I appreciate the conversations we have around regulation. I shared many of these when I was working during the Obama administration. These things I can share publicly I put out in an open letter to the new administration about how we can approach this regulation. I can lay out a couple of those issues.

The first easy one, just for people to visualize not necessarily to implement, is separating the data from the data model. You know, we can have bias in the data. We can have bias in the data models, the algorithms. But separating those out can give us some purchase and then thinking about what we want to have happen.

The data that we collect, think about an easy example, where Delta airlines, where I often find myself, will know my height, and my age, and maybe where I fly from

and to. But then they have their data model that may determine some configuration of that data for their election to give me some other sort of benefit, upgrade, call it. Those separations, I think, are going to be really useful in where we want to regulate.

Angelo: Machine learning is not the only method. Simulation is another branch of computer science that deals with trying to build a computer system that can simulate or model the real world. I had back, a dear friend of mine, Andy Lee from Vincere Biosciences, who's built simulation technology for simulating chemical compounds in the human body.

I did a little bit of research myself in simulation. I've done some simulation in various ways. We did some clinical decision support tools and I'll talk about those later, but simulation is such a big, broad topic. There is simulation done, I read a great paper on simulation in surgery where what they do is they take fluid dynamics and try to compute blood flow in vessels. And then how the vessels would react in surgery. That's one part of it, I think, simulating the body and how the body would react to things. But then there's probably simulation on the chemical structures themselves. So tell me a little bit about what simulation, you know, what are you guys simulating and how have you kind of found that effective.

Andy: You know, one of the challenges there is figuring out what level you are going to simulate. Like how deep do you go? Because you could go down to the energy minimization at the protein structure and chemical structure and docking simulation. And we do some of that at certain points of our project, but then our core is trying to scale that and find a happy point in the middle of biology simulation.

And, you know, you could go the other end of it and just do reaction rates and systems biology type modeling where you're, not getting into the physical aspects at all. So finding that point in the middle was an interesting challenge.

Angelo: Yeah. So when you do biology simulation, what are you simulating?

Andy: Really looking at the, you know, if you think of a cell, most of what we're doing is simulating a single cell. And then looking at the interactions that happen inside of that where you've got quantities of proteins that have a certain properties, they have a mass and a size, and then the transcript Omix that you can measure from patients can give you the quantities of those.

So you pack them all into a three-dimensional space that is defined by the observable shape of the cell and the components inside the cell. And then using physics to play that out over time, you can then kind of model the movement of

those proteins within that cellular environment and then layer in interaction rules that can drive some of the state transitions.

Angelo: While there are undoubtedly challenges and we talked about a bunch of them, there's also a lot of opportunity and that's how I want to continue from here on out in this episode. Thinking about why we actually do this, computer science, despite all these challenges, has tremendous opportunity to be able to unlock potential, to be able to unlock things that we might not have been able to understand otherwise. Astronomy is one of those things that would not exist in its modern form if there was not computer science. We just could not possibly process the kind of information we have. In episode two, I welcomed one of my colleagues here at Ballista, Paolo Bonfini, an astrophysicist now turned computer scientist himself, to ask a little bit about his thoughts on this problem.

Paolo: Well. Let’s say that there are a lot of computations involved in this. People have in mind the romantic view of the astronomer who just looks through the scope of the telescope and, you know, notes things down. But modern astronomy is completely digitalized.

And recently it has been even automated by a lot of procedures that they track and scan the sky to create huge catalogs. Even the images themselves, they are captured on digital devices, like, the same as they appear in the phone, basically, the same technology, but just on a more refined scale.

And the first process for which you will need a computer is to combine exposures. So you cannot expose a telescope on a specific direction in the sky for a very long time, for several reasons. The summary is that, in order to take an image of some patch in the sky, you will have to take multiple images and then combine them.

Now the modern telescopes, they are extremely accurate. So when you combine them, you need to align the stars to a sub pixel resolution. That means that you have to find the center of the star and itself be positioned within a single individual pixel. And when you combine images, you have to align them by with the precision of, let's say, a third of a pixel, which sounds impossible because you're like, how can you do that?

But, there are some techniques that allow you to do that. And of course you need a lot of computational power for that. It can take several minutes to do this, even a half an hour, let's say, to combine and produce the image that you see on famous websites, like the Hubble.

I mean, this is just the first step, you mentioned. Then you need to actually extract, in my case for the study I was doing, in order to assess the lack of stars at the center of a given galaxy you actually have to measure it.

Angelo: There were some really powerful concepts this season. We're seeing the emergence of things that we talked about like AI. We’re going to talk about quantum in the future. We see these technologies emerging, but really what I want us to understand here is that in our journey, through improving computer science, getting more and more computing power, which has really changed the way that the world works over the last few decades, we’re going to encounter the same thing that we're encountering now. So Moore's Law, at the beginning of computer science we were doubling every 18 months in computing power, and this is golden age of being able to do more and more things at such a voracious rate. But then after a while we started saturating it and we're kind of getting at the end of it.

And we're trying to squeeze every little drop out of performance now. Now we have to rethink the problems and reframe them in different ways in order to be able to continue to innovate. But something like quantum comes along and completely reshuffles the deck. And we are going to start all over again in a golden age of quantum where things are going to be some other law, like Moore's Law, that's going to be able to have this amazing improvement.

Think about it. Qubits are the unit of computational measure in quantum. A qubit, like a bit in a regular conventional system, we only can do so many qubits now. But eventually we'll have hundreds or thousands of qubits, and then we'll start to saturate the way that that works. And then it's Moore's Law that corresponding whatever it will be will come into play and we're going to have to ring out every little bit of other things that are going to make it work. Some new ideas.

All of the ideas we talked about, even thinking back to the AI winter, all of that started a very long time ago. Right? And those ideas now are being recycled. Columnstores are not brand new ideas, they're decades old, but then they went on the shelf because computing power was just doubling and computing wasn't our issue. It wasn't our bottleneck, and then it was, and now it's not again with memory and then it won't be again, and we're going to have these recycled problems. And what I really want us to get out of this is we're going to be doing this. And we're going to be doing this again and again and again, no matter how many times we have a paradigm shift that gets just dropped on us.

And that might sound defeatist, but it really isn't. It's actually what makes computer science so exciting. There will always be a challenge. We're never going to reach the end of what computer science can do. We're going to always have these innovation cycles and invention cycles and paradigm shift cycles.

And we get to apply new knowledge and then there'll be this long forgotten knowledge that someone will dust back off and prove that it works again. And I think that that's what makes what we do so awesome.

And that's why I think we are so lucky to be in the time period we’re in now. Think about it, computer scientists right now have this palette we're putting together and have been putting together, so you get to innovate, you get to write papers that are going to be used as a foundation for a very long time. And then brand new ideas that we're just on the cusp of things like quantum, that frankly, there are not many experts on, gives us another opportunity. So we're going to be able to branch out in so many different ways to be able to work on so many problems, that it's just super exciting. And I'm glad I'm part of it.

And as we close season two, I want to thank all of our listeners for supporting us. And I hope you really enjoyed season two as much as I enjoyed being a part of it. We look forward to having a great season three and of course, we look forward to your feedback. There's some really interesting and cool things coming that I think you're going to be excited about.

And so as I close this out, I want to say, thank you again. I'm your host, Angelo Kastroulis and this has been Counting Sand. If you haven't already, please take a second to follow us and to connect with us on social media. You can follow us on Twitter. My company is @BallistaGroup or @AngeloKastr, me personally, or you can find us on LinkedIn @AngeloK1. Thanks for listening.