Angelo talked last season about computer science making our lives better. And this season, he is going to talk about what are ways in which we can bring new technology forward. In this episode, Angelo talks about the ending of Moore’s Law and what is the next horizon of research acceleration to practical application.
Angelo begins this episode with the predictions of Moore’s Law. In the early years, systems were restricted based on the CPUs ability to keep up. As the CPUs continued to advance, the bottlenecks ended up around data movement. Data movement of information from disc to memory and memory to cache become the big bottlenecks. Then of course, disks got faster and eventually you'd have so much RAM on a machine that it was just memory movement inside of RAM. Eventually, we believe the bottleneck will return to the CPU.
Quantum computing is on the rise—that, we believe, is a game changer for Moore’s Law. Because we're no longer talking about conventional computer chips and transistors, instead we're talking about something completely different.
Additionally, as machine learning and artificial intelligence systems make advancements, or like Angelo’s thesis using AI to tune data systems, the advancement of speed and acceleration will be impactful in order of magnitude from traditional systems.
Talent and staffing will also change as we adapt to the future. Angelo admires Google’s practices of hiring ability over experience because the problems we face tomorrow are different than today. The key thing is to be able to independently make progress because there isn't much room for babysitting. It's too hard to predict where the next fire will be. Angelo explains further why he hires ability over experience every single time, because it is true, someone who has ability, someone who's brilliant and has the hunger to learn new things can be programmed like a stem cell. They can just inject themselves into whatever problem they might have.
Angelo transitions into his own personal story and his quest for fulfillment and happiness. He introduces a personal story of a boy who was dying during the Nazi occupied island in Chios, Greece. A doctor took pity on this boy and secretly nursed him to health. We later learn that this boy is Angelo’s father. Angelo shares, “My father grew up in a world much different than mine. His siblings related stories of famine and suffering, but he never ever spoke of those things. What he chose to relate were accounts of human triumph, perseverance, hope, aspiration. The sea was his salvation, carrying him from Chios as a sailor, eventually to the United States.”
So, what is our true potential? Intellectual achievements can be ignored or forgotten. But to be a successful family person, a husband, a father, a human, Angelo needed to be something more, something enduring.
Education builds the qualities of perseverance, hard work, and accomplishment. There is no doubt you'll accomplish many things, but think about what it is that you're really trying to do. You see, building technical solutions isn't just about doing interesting stuff. Ultimately we're building these things for a reason. We're building technology. For example, if you're doing a healthcare application, it's going to touch somebody's life. That's the point of this breakthrough, right? You want to increase throughput, for example, in decision support, something Angelo spends a lot of time on. We want to say, increase throughput, build a system that can compute faster and bigger sets of data. Why are we doing that? Just because of the challenge of the data? No, we want to find out if a clinical intervention is working so that we can feed that information forward to those making the guidelines.
You see, that's the real reason behind doing this. The great resignation has shown us that people care more about what it is they're doing and why they're doing it than just simply being interesting work. We owe it to our family to use our gifts, talents, and opportunities to the best of our ability, but to use them on something that matters.
Angelo is really excited that we're going to have interesting conversations around things like the universe, data centers, energy and how they work. There's a reason the hard problem exists. Don't fixate on the fact that it's a problem. Although there is joy in having a problem and solving it.
We're trying something a little bit new this season and we would love to hear which kinds of episodes you like most. Do you like interviews or do you like some of the educational discussional episodes?
We're going to start a YouTube channel to help deep dive on topics like LSM Trees or RocksDB which are better served with diagrams than with just voice. Seeing the math for yourself or seeing the way that they operate for yourself on video is much more helpful. We're going to have supplementary content, bonus material that you can find on our YouTube channel, and we'll also have some bonus podcast episodes. We look forward to your feedback. Tell us what you like about the show, which topics you prefer, and what you wish we would dive a little deeper on. And we'll really try to do that.
Citations
Gordon Moore, Co-Founder of Intel
Heisenberg, Uncertainty Principle
Powell, James. (2008). The Quantum Limit to Moore's Law. Proceedings of the IEEE. 96. 1247 - 1248. 10.1109/JPROC.2008.925411.
Merritt, Rick. (2013). Moore’s Law Dead by 2022, Expert Says of EETimes
Atomic Hire (2019)
Further Reading
About the Host
Angelo Kastroulis is an award-winning technologist, inventor, entrepreneur, speaker, data scientist, and author best known for his high-performance computing and Health IT experience. He is the principal consultant, lead architect, and owner of Carrera Group, a consulting firm specializing in software modernization, event streaming (Kafka), big data, analytics (Spark, elastic Search, and Graph), and high-performance software development on many technical stacks (Java, .net, Scala, C++, and Rust). A Data Scientist at heart, trained at the Harvard Data Systems Lab, Angelo enjoys a research-driven approach to creating powerful, massively scalable applications and innovating new methods for superior performance. He loves to educate, discover, then see the knowledge through to practical implementation.
Host: Angelo Kastroulis
Executive Producer: Náture Kastroulis
Producer: Albert Perrotta;
Communications Strategist: Albert Perrotta;
Audio Engineer: Ryan Thompson
Music: All Things Grow by Oliver Worth
Angelo: Welcome, and I am so excited to welcome you to season two. We've got some really interesting things we're going to be talking about really, we're going to be talking about really what's not in the very far future, but what's next. We talked last season about computer science making our lives better. And this season, we're going to talk about what are ways in which we can bring new technology forward.
Now, we will sometimes talk about the distant future as well, but we want to again, continue the theme of solving hard problems today. My name is Angelo Kastroulis, and this is Counting Sand. I want to talk just for a second about Moore's Law. You may remember Moore's Law was an observation that Gordon Moore made in 1965. He observed that the number of transistors in an integrated circuit doubles every two years.
Just to put that in context. That's 1965. In 1971, the state-of-the-art was 10 micrometers. That is 10,000 nanometers. And in 2022, we're in a three nanometer process. We are being projected by 2024 being in the two nanometer process. But we can't go to zero. So either way, there's not a lot of room left.
George Powell [correction: James R. Powell] calculated that due to Heisenberg's uncertainty principle alone, Moore's Law would be obsolete by 2036, but as you've seen from some of these numbers, we've just talked about 2036 is way later than when it really will be gone.
Robert Caldwell, who used to work at Intel and was the chief architect behind the Intel Pentium chips between the Pentium Pro and the Pentium 4, later became director of the microservices technology office at DARPA. And he said back in 2020, that by 2022 Moore's Law will have ended because of the five nanometer process.
Well, why do you say that? Because we could still go a little further. Well, the three and the two nanometer process and who knows? Maybe a one, I don't know. Three and two nanometer processes have not yet been proven to show such a substantial improvement over five.
That was his argument. That three may only be marginally better than five. But you'll pay a bunch more for it. So then Moore's Law would have come to an end further before that. So it isn't necessarily tied to the wafer process.
Now it used to be that CPU was the bottleneck of computing systems. So a lot of our algorithms track that the algorithms of the day that started winning were the ones that were efficient with CPU. And the ones that dealt with data movement were kind of put on the back shelf. Then over time. It shifted to where the CPU, just through Moore's Law, kept improving so much so that they were outpacing data movement.
And then the movement of information from disc to memory and memory to cache were the big bottlenecks then of course, disks got faster and eventually you'd have so much RAM on a machine that it was just memory movement inside of RAM. That became the problem. So the problem has been kind of moving, but eventually that will become so improved that we'll be back to CPU being the bottleneck and the fact that we have no more room for improvement, can't make it any smaller. Will make that bottleneck insurmountable. So what did we do?
Well, that is what we're going to talk about this season. What kinds of things can we do to have improvement despite the fact that we've been relying on computing power for so long? The other thing I want to point out about the number of transistors on a chip is that over time, and I talked about this with some colleagues on an episode we recorded which you'll hear soon, is that computing power on these chips has led to vast improvements in computability, but the number of transistors going up, if you're actually looking at the cost per transistor, because of the small size and the amount of technology needed to make it small, the cost per transistor is going up significantly. For example, the five nanometer over the seven nanometer is 50% more expensive.
So the number of transistors do not outpace the price. So it's actually going up substantially. In other words, we could have bought a transistor years ago for much, much less than we're buying them today.
There are other more forward looking technologies like quantum computing that while they're not quite there yet, they're still early. Maybe they’re, depending on who you ask, five to twenty years away from being usable. They still have some impact now. And it also depends on the kinds of quantum technology we're talking about.
That being a game changer in the future is going to be a way for us to be free of Moore’s Law, because we're no longer talking about conventional computer chips and transistors. We're talking about something completely different.
Additionally, continuing themes, like machine learning and artificial intelligence to make advancements, or like my thesis using AI to tune data systems, using that to make things faster than others to use what we have better. That's some of the technology that you'll see coming up now, we'll also talk about things like privacy. We've talked about that in season one. We're going to continue talking about that. How can we reduce costs? How can we improve our situation like health care? How can technology help us to understand the universe? Those are some of the really interesting things we're going to talk about the season.
One of the themes I would like to continue, or I should say one of the threads I'd like to continue is how do we shorten the gap from something theoretical to application? How do we take this research from academia and put it to work? So from breakthrough to reality, it takes us sometimes two decades to get there.
Rarely does it happen immediately. How do we shorten that gap and what is causing the gap? Why does it take so long? Well, there's a lot of things involved, of course, but I want to talk about one factor and that is fear. Research isn't for everyone.
Not everybody naturally takes to it. I think that one of the reasons that, many engineers fear looking at the cutting-edge papers and pulling them in is because a lot of times risk is rejected in most of our jobs. We're we penalized for taking risks that might increase costs, right?
Because we're so constrained by the budget. And so the safe, predictable route, the thing that we can always count on and measure in our estimates, being very on all the time, is the one that's constantly encouraged. But that one is going to eliminate any innovation. And so I think fear of failure is one of those things, finding the breakthrough is hard and for every breakthrough there were hundreds or even thousands of dead ends.
So for the seasoned technologist. Fear of failure causes us to take the safe route, or maybe we don't even have the opportunity to do it because how could you pitch it? If you're just starting out in the field or you're looking to improve yourself, you know, a veteran who's maybe looking to improve your ability, I have this advice.
Position yourself as someone with ability. Experience is overrated. Urs Hölzle, one of the early Google technology leaders said this: "hire ability over experience.” Brilliant generalists can reprogram themselves like a stem cell. The key thing is to be able to independently make progress because there isn't much room for babysitting. It's too hard to predict where the next fire will be. And I couldn't agree with him more. I hire ability over experience every single time, because it is true, someone who has ability, someone who's brilliant who has the hunger to learn new things, can be programmed like a stem cell. We can just inject ourselves into whatever problem we might have.
And yes, we aren't going to be the expert in it. But if the expectation is not that we're going to be the expert in it and do this extremely efficiently and cost-effectively, but we're going to end up with the best solution in a reasonable amount of time, you know, learn a new programming language in a week for example, that's not impossible. We do that all the time. If you're trying to find a role to settle into and to coast, are you really going to be happy? I'm going to say that I don't think you are. In whatever you do, I urge you to seek happiness. While I think you can find fulfillment in your career. It's not really going to be your source of happiness. Fulfillment and happiness are different.
I find fulfillment in my career and I do find some measure of happiness in it as well. But start by figuring out what's important to you.
I'd like to tell you a story that I think maybe illustrates that point. In 1943, a five-year-old boy was suffering from malnutrition on the Nazi occupied island of Chios, Greece. A German doctor realized that his condition was much worse than the boy thought. The boy was dying. Moved with pity, he instructed the child to meet in secrecy every day before daybreak. The boy and his mother came each morning faithfully for the next few weeks for treatment and nourishment. He gave him a shot and a piece of bread. Over the next few weeks, the boy got better and he survived. The doctor by all accounts was highly educated and the boy by all accounts was not. The boy would not see the inside of a school house for his entire life.
No doubt, the Hippocratic oath the doctor took, played a role in his thinking, but was that his entire motivation to act? The boy was my father and I would not be here today if it wasn't for the kindness of the one man who in the face of tremendous risk showed love. I'm one consequence of that moment's resolve, but it profoundly changed that boy forever.
He adored the sea and spent as much time in it as he could. Opportunities in post-war Chios were hard to come by and perhaps in a different era, he may have realized more from his talents.
He was an accomplished swimmer and extremely intelligent. The world my father grew up in is a world much different than mine. His siblings related stories of famine and suffering, but he never ever spoke of those things. What he chose to relate were accounts of human triumph, perseverance, hope, aspiration. The sea was his salvation, carrying him from Chios as a sailor, eventually to the United States.
He always dreamed of a higher education, not for himself, but as an investment in the next generation, a dream for a better life for his children. As a father, I completely understand. We realize our dreams through the generations that follow us. As the first to go to college, I grew up with that spectre of those ambitions.
I imagined that education was required to live up to all that hope. I was wrong. I started to see flaws in my reasoning when I considered the qualities I most wanted to emulate. It was not famous scientists. It wasn't Alan Turing, it wasn't mathematicians that struck me with awe. Although, I am certainly impressed with their accomplishments.
Instead, I was astounded at the ability to see the worst in humanity yet choose to set it aside and focus on the best of it. I saw triumph, perseverance, unbreakable will to survive in such a world, unscathed by hatred and still see the best in humanity. To me this seemed superhuman. I also pondered the example of that doctor.
An oath is of no value without the strength of character to act. If we don't dare to do what's right, of what value is all the education in the world? A wasted oath benefits no one. But the more I search, the more I find examples, just like that in ordinary life, their sources firmly rooted in the quality of love. In time, of course, these men died yet what endured? The legacy that they left by touching the lives of another, that moment of that doctor touching the life of a young child had far reaching consequences for generations. I wonder if he ever pondered that whatever happened to that boy. That boy would also affect many lives leaving the world unimaginably better than he found it. That's real lasting success.
Now I get the realities of life. Continuing education in my forties has been a difficult. That's when I went back to grad school, it required resolve, balance between raising children, earning a living and maintaining your sanity. That's not a unique circumstance. I know many people do that. We all want a better life for our families. John Wooden, one of the most revered and successful basketball coaches of all time put it this way, "success is peace of mind, which is a direct result of self satisfaction in knowing you did your best to become the best you are capable of becoming." So, what is our true potential? Intellectual achievements can be ignored or forgotten. But to be a successful family person, a husband, father, human, I needed to be something more. DSomething enduring.
If we fill the void of struggle and suffering with love, we realize our true potential. I know that our parents have many aspirations for us, but I doubt that the motivation behind every single one of them is not backed by love. Ask any parent to describe the profound moment they perceived love and usually they'll tell you, it's a tender moment like when you held your child for the first time. The Greek language has many words for love, but we must experience them all in order to be understood. Now that is the most beautiful part of those words. Every single one of them needs to be experienced.
That's not to say that education is of no value. Could education give you the ability as John Wooden said to have this self satisfaction in knowing you did your best to become the best you're capable of? Sure it does. Sure. It plays a part. But education is just the beginning. That doctor would not be in the position to make the difference he made without the education.
Education builds the qualities of perseverance hard work and accomplishment. There is no doubt you'll accomplish many things, but think about what it is that you're really trying to do. You see building technical solutions isn't just about doing interesting stuff. I will admit that has a measure of happiness to it in and of itself.
But ultimately we're building these things for a reason. We're building a technology. For example, if you're doing a healthcare application, it's going to touch somebody's life. That's the point of this breakthrough, right? You want to increase throughput, for example, in decision support, something, I spend a lot of time in.
We want to say increase throughput, build a system that can compute faster and bigger sets of data. Why, why are we doing that? Just because of the challenge of the data? No, we want to find out if a clinical intervention is working so that we can feed that information forward to those, making the guidelines. You see, that's the real reason behind doing this.
And the same is true in almost any field. The Great Resignation has showed us that people care more about what it is they're doing and why they're doing it than just simply being interesting work.
I want to bring out one more point on this before we move along. As a parent, I know an awful lot about my kids because I was there to witness all of the moments in their lives. But my children really only witnessed a very small portion of my life and only windows at that. I have an entire segment of my life, for example, my career, that my kids know very little about.
I also had an entire lifetime of events before they came along. I thought about the same thing for my parents. The only things I know about their past are the things that they chose to share with me or others chose to share with me, those few moments that I wasn't consuming their entire attention and focus.
I was the center of their life. So I wasn't reciprocating the kind of attention they were giving me. As I continued digging though, I found that my father, for example, had been in the middle of all sorts of really important historical events, almost like Forrest Gump had been. Digging further, I learned that one of my great grandfathers lived in the United States.
He spent much of his life working on the railroads for Vanderbilt. Then some event, which nobody seems to know occurred, which caused him to give up everything, move back to Greece, and live primitively as a farmer, forbid all his family from going to grade school. What made him do that? Each successive generation worked to move the bar after that further and further so that their children could have a better life.
And each did slowly, progressively until we came to my generation where I was the first to have an opportunity to do anything I wanted to do. If I wanted to work in computer science, I actually could. The weight of that hit me. I'm the culmination of generations of perseverance and hard work. So I have a responsibility to those generations not to squander this opportunity I was given by wasting it on something selfish.
After all, did all those people spend their lives building themselves and the next generation forward, just so that I can lay on it. I try to impress the same thing to my children. We owe it to our family to use our gifts, talents, and opportunities to the best of our ability, but to use them on something that matters.
So figure out what matters to you. If my father taught me one thing, it was that "the why" matters. And I will say people matter too. So remember, one of the keys of happiness is the lives we touch. When we're gone, that's what will survive us. Research breakthroughs, while they are immense, exhilarating, fulfilling accomplishments, that they touch no lives, they're not going to be remembered.
There's one more thought I want to share. It was given by Harvard president (Lawrence) Bacow during his speech on my graduation day. I don't remember the exact phrase, but it's something like this. He said, today is a day for you to rest on your laurels, but only for today. Tomorrow, go out and change the world.
I think I added the last sentence, but it's the feeling that I took away. It's a reminder that it isn't necessarily about changing the world. We all won't be able to do that. Although we can certainly contribute to it in our own way. It's about living in the moment.
Sometimes you have to stop and do that, or else those moments that you should savour will just pass you by because you're on to the next thing.
I'm going to admit that that is one thing I historically have not done well. I just immediately move on to the next thing on my list. Great, that's done. When you accomplish something, take a moment to enjoy it. Pause and reflect. That reflection will bring a feeling of gratitude and gratitude, my dear listeners is the other key to happiness.
That's what I would add to his speech.
So, as I mentioned, we're going to talk about some cool things this season. Gratitude is an interesting topic because it hit me when we were recording one of the episodes that you're going to hear soon, about detective galaxies and space-related stuff using AI. But one of the things we talked about off-camera was how some of the cosmological models are being effected. As you know, the universe is said to be expanding and so we can run simulations to see as it's expanding, it's actually accelerating. So the galaxies are moving away from each other faster and faster. But as it turns out, the models are showing not only are the galaxies accelerating, but so are solar systems from their galaxy and planets from the solar system and even matter. The atomic pieces are moving apart. So if you follow that out for 10 or 20 billion years, first of all, all of the stars will burn out. They'll eventually run out of gas. That's what they're showing and it'll become darkness in the universe.
And then of course, as everything continues to expand, eventually matter will disintegrate and there'll be nothing. Okay. That sounds really, really sad. And I'm not trying to be a downer, but in 20 billion years, there'll be nothing. So think about infinity after that will be nothing. And infinity before the beginning of the universe, there was nothing. And now, only this pinpoint in time. Okay. Yes. This pinpoint in time that we live in is a very long period of time in which the universe, as we know it, exists. But in the period of infinity, you are astronomically lucky to be alive right now. So gratitude is really interesting when you think about it in that way.
Now, of course, those are models and who really knows, maybe there'll be some cosmological intervention, right? Or maybe that's not how the universe works, maybe stars rekindle, who really knows. But it's just interesting how our understanding of the universe and AI and software helps us understand it, right?
Without simulations in AI, we wouldn't have any kind of understanding of what these things might look like. It's more than math, I guess, is what I'm trying to say.
And I want to point that out too, that I think that what's interesting about this season is it is about technology, but there are so many profound side effects to that. Things like that, that you might've never thought of before.
But back to our topic of what we're trying to accomplish in this series, I'm really excited that we're going to have interesting conversations around things like the universe, data centers and how they work. Energy. There's lots of really interesting stuff coming. There's work being done all over the place and I focus a lot in healthcare. So I've talked about healthcare historically. We're going to continue to talk about healthcare and the impacts of software on healthcare. The point is that, the stuff that we work on has some meaning. Now that doesn't mean that everything we ever work on is going to be intellectually stimulating and on the edge and have meaning.
It just means that as you solve the hardest problems, there's a reason the problem exists. So don't fixate on the fact that it's a problem. Although I will say there is joy. I think I've said that before there's joy in having a problem and solving it and just focusing to solve this one particular slice of a problem and accomplishing that.
And then that's in fact how most research papers are, right. They don't boil the ocean, so to speak, or try to solve the world's energy crisis or whatever. You're trying to solve one particular small thing. Like for example, my thesis was on using artificial intelligence to determine the access path of a scan versus an index.
Now it grew a little bit, but it didn't grow so much to say, hey, I've got this new kind of database I'm proposing. I guess you could do that as well, but it won't grow so big as to solve all the data system problems in the universe. It doesn't work like that. But a lot of times when we think about the applications of these, they have bigger meanings.
So while I say, thinking about those things, it's important, looking at the problem too is also really, really important. So I'm excited that we're going to have opportunities to do so. We're going to talk about some other things in depth and we would love to get your feedback. We're trying something a little bit new this season and we would love to hear which kinds of episodes you like. Do you like interviews or do you like some of the educational/discussional episodes?
So please feel free to give us your feedback, but we're also trying something a little bit new with video. We're going to start a YouTube channel to help deep dive. Because it makes sense. So when we started talking about last season, LSM trees, for example, we talked about RocksDB. We'll talk about more about these things as we go, but they're very hard to talk about on a podcast.
You need a little bit of a visual representation and then seeing the math for yourself or seeing the way that they operate for yourself on a video is much more helpful. So what we're going to do is have supplementary content, bonus material that you can find on the YouTube channel, and we'll also have some bonus episodes.
So we look forward to your feedback. Tell us what you like, what topics you like. What kinds of things you wish we would dive a little deeper on. And we'll really try to do that, you know, kind of true to our own selves of researching and the practical side of it. This is the research part of it. And to being a data scientist, we want to see the data and see what makes the most sense for you.
Thank you for joining us today. I'm Angelo Kastroulis. your host, and this has been Counting Sand.
Before you go, please take a moment to like, review, and subscribe to the podcast. We're also going to launch the YouTube channel we mentioned so keep an eye out for that. You can follow us on social media. You can find me at LinkedIn at AngeloK1. Feel free to reach out to me or start a conversation.
You can also follow me on Twitter. My handle is @angelokastr, or you can follow my company, Ballista Group.
You can also find the show on countingsandshow.com or your favorite podcast platform.