Angelo and Shohaib discuss the process of introducing new ingredients into hot sauce production using AI. They explore different approaches, such as creating new models or expanding the feature space. They also discuss the concept of transfer learning and whether to build upon an existing model or start from scratch. They highlight the benefits of using metaphors, like hot sauce, to explain complex concepts in a more tangible and relatable manner. Shekeib, who is less familiar with the technical aspects, appreciates the ability to understand and learn through these analogies. The conversation touches on the broader field of AI, including topics like Bayesian optimization, neural networks, reinforcement learning, and generative pre-trained transformers. Angelo expresses excitement about discussing mathematical concepts in a way that can engage a wider audience.
Introduction:
Incorporating New Ingredients:
Metaphorical Understanding:
Engaging with Mathematics:
AI as an Expansive Field:
Reinforcement Learning:
Specialization and Continuous Learning:
Generative Pre-trained Transformers:
Angelo: So in this last batch you sent us, there are some interesting things in here —carrots and onions in one of these. Discovering new ingredients, is that part of the process or do you kind of just create new models for it? How did you introduce new ingredients? Shohaib: So you'd have the optimized version of the five ingredients and plus the newer ingredients. So those are not optimized yet, but this is, say, iteration one of those hot sauces. So that's what's so exciting about that. Now we're adding a couple of more ingredients, having that search space, but limiting the search space of the new, maybe it can move around a little bit. That's what I’m thinking. Angelo: So let me ask you this. And you don't have to have solved all this, but are you thinking of just increasing the feature space to add more features or are you thinking of doing something like transfer learning? Where you say, okay, we're going to keep the base model as it is and we'll create another model for these extra ingredients to create variants? Have you thought about that? Shohaib: Say you have the optimal ingredients and then now we want to transfer it into another model. I think it would be easiest to keep the model, increase the search space on the two ingredients coming in. But I thought about it maybe both ways and even in the sense that since we have the optimal feature we can start with even just these two hot sauces and go from there. Angelo: This is great because, again, it lets you conceptualize really hard topics and we can just talk about them in the context of hot sauce as a metaphor, as a stand in for like the more complicated concepts. And they all make total sense. We can all talk about them. So the metaphor would be, well, if I was making it myself in my kitchen, do I include this base and then try to figure out what the other ingredients are by keeping it stable? That's transfer learning. Or do I start with a whole new recipe considering all these? That's one of the things I love. It's so tangible. Shekeib: I think my brother was trying to explain that to me pretty recently. How to add new ingredients properly. But the way you explained it made me understand it even more. So that's just the great thing about this because what you guys are talking about would be over my head. But since I can relate it to the hot sauce and everything, it makes me feel like I'm learning so much more than I would, you know. Angelo: Well, I love it too. Shohaib, I don't know about you, but I have fun talking about the mathematical side of this and I get just as excited. But it's nice now that other people can kind of join me in my excitement, you know, the hot sauce side of it and we're all kind of like enjoying it and having fun together. Whereas when it's just people talking about math, everybody else kind of feels not part of the conversation and they're not as excited and they say, how can you be excited about math? Well, if I call it something else, all of a sudden you're excited too. So I think that that's pretty cool. Shekeib: Yeah. And then the great thing is you can apply it to almost anything. So it's, yeah, you can get people interested in AI in so many different ways. But yeah, my brother would love to talk all day about math. He'll think about stuff and he'll start writing on a piece of paper, different equations, and I'm like, man, it looks like he found somebody who might be interested in talking to about those things. Because it just goes way over my head. So it's cool that you, you guys enjoy that like at that level. Shohaib: And to your point, this is kind of a subset of a, I mean, we're talking about AI, which, encompasses machine learning, which encompasses Bayesian optimization, active learning. So that's kind of what we're talking about, but we're able to conceptualize it in a hot sauce, which is, I think is absolutely amazing. Whereas maybe with machine learning, you have maybe the optimal weights, so once you have the optimal weights, then that's your optimal, whereas with Bayesian optimization, you're adding to the data set and then you’ll optimize later down the road. Angelo: Yeah. Well, I'm glad you said that because that is a really deep kind of concept here that when we talk about AI it's this gigantic umbrella. Bayesian, which is the thing we've talked about today, is one type of machine learning approach. There are neural networks. There's a whole other thing that's not really even machine learning, but it could be. We haven't figured out what we want to call it. We could call it deep learning. I don't know. So there's that, and then reinforcement learning was kind of neat because people figured out how to solve video games from it and they kind of like now can be applied to a million things. There's so many things. Shohaib: There's a lot of similarities between concepts in Bayesian optimization and reinforcement learning, which is an extremely, extremely powerful tool. That's one of the areas that I haven't applied a lot of things in. But we've done reinforcement learning in an AI class, I think in a Pac-Man game to make the PacMan play by itself, so. Angelo: Yeah. That's what made it famous. But it is also another method of getting to a really good answer when you don't have a lot of data or a priori knowledge or time to train or whatever it is. It's just another way and I love that we're building these and it just goes to show, you can specialize in this one thing and still not even know everything there is to know about it. You've settled on Bayesian becoming an expert there but there's, you know, you have two master's degrees and there's a lot of things that you know, but in order to kind of get to this path, so that's pretty awesome. I wanted to ask you guys, you know, yet another kind of machine learning, which is like super famous and popular, all of a sudden, is generative, pre-trained transformers or, like ChatGPT. Yet another kind of machine learning, which is interesting because it's an ensemble of some different techniques put together in a new and interesting way. I just wanted to hear if you guys have any thoughts, have you played with them? What do you think? Shekeib: Oh, my brother definitely has. I'll let him talk about it because he's got a lot of good information on it. But just seeing the things that he can ask the ChatGPT and the answers it comes up with on uncomplicated things, it's just really impressive. The problem my brother's had is sometimes it's confidently incorrect, but it's definitely been a really useful tool for him. So I'll let him go off, but I would think he's got some of the most interesting ChatGPT conversations out there. Shohaib: Well, I mean, for someone that loves mathematics and wants to get under the hood of all of this, it's a great learning tool. I mean, I would ask it various things, but why does a Gaussian process have this type of dimension in its predictive variance. Or things like that, just specific machine learning questions and it's almost like you have a professor. You know, certain things it may not get correct and then you can ask it certain things to where you can get down to exactly what you want to ask, and then it'll learn from past responses as well. And so, I feel like it really is like having an educator there with you and if you have a professor there, it'll get tired of answering questions, but if I'm constantly getting answers and interesting answers, oh, if that's really how it works then how could I do this to, you know, do this or I don't know. There's so many different avenues but for me int’s been a great learning tool. Angelo: Yeah I think that, you know, the way I started with it is, I started just playing with it and I said, okay, I want to just test its limits. I want to understand what it does. What I was trying to do is see if I could figure out what it's trying to do or how it works or whatever. And then, as I was going through that, I was baffled. You know, where I looked at it and said, okay, how does it semantically understand what I said so correctly? Okay, that's one question. And then how did you produce and generate this output so correctly? I mean, the two things had to link so well. And then you start tearing it apart and getting under the hood and trying to figure out what it does and starting to see those pieces. And assembling all the parts that it did, all the different techniques that it did to get that—it’s phenomenal. Now, I will say the code it generates, don't rely on it, but it definitely gets you started it. And it starts the initial process of discovery where you go, well, yeah, what's the state of the art of this? Or what are the approaches for this? Then you give it feedback and you start fixing the code and you give it back to it and say, now generate this that does this and write tests for this. And it can do it at that point, but it's not, not going to replace people, but it's a great assistant. Shohaib: It is definitely a great assistant, you know, working with it and having it produce some code. Now it’s gotten even better but sometimes whenever I worked with it in the beginning it would forget libraries or it would square something and then square it again. It's like, why did you square it again? It's like, oh, I'm sorry, I messed up there. It's like, you almost kind of want to get down to why it messed up there. Like if everything else was right, where did it go wrong? Like, you kind of want to go under the hood. And so I've briefly looked into that. There’s some reinforcement learning in there, and there's been some that have been labeled by actual people or they've trained it themselves, you know, AI experts. So it's not completely unlike the hot sauce in that aspect. But like you said, it's an ensemble. It's not something that I've looked into. I've just enjoyed trying to learn as much as I can in various things or, you know, it could actually even give you tasks to make your programming ability even better. But it’s almost indispensable now. Angelo: That's the same outcome I've come to is I can't live without it anymore. It has made me so much more productive than I was before. In the very limited time you have, you're able to spend it in a way that that can get you the most productivity. Has it replaced a lot of things that I do? I mean, repetitive things. It's helped me in some of the research, you know, the things I used to search on the internet for and spend a half a day searching. I can get a quick answer, I can ask it, what are the references to this and then I can go read those papers instead of, kind of, let me fumble through and find papers with keywords. So some of that is even great. Shekeib: Since we're making this hot sauce and Náture, I hope I'm saying it correctly, she expressed to me that, do you guys have some social media links? And of course I've got social media links for my own, so I started making some for the hot sauce, and I was like, how can I sell this hot sauce or what should be the, you know, type of text I should have on my Instagram post? I was like, let me see what ChatGPT has to say. And I would just type, can you make a description for a hot sauce with these ingredients, this type of flavor, and make it for the Instagram crowd, or make it for the Twitter, you know, it understands what type of dialogue works best on those. And it made a really great, you know, and I edited just a little bit just to make sure it made sense, you know, but it was amazing. I’m doing the same thing with some of the labels. I bought a subscription to Midjourney bot. It's basically an AI generator for images. And I just described what I'm going with for the hot sauce. I can even show you a picture of one of the ones it made, I think it shares here, the entire screen. Yeah. And you can see it makes a really great looking hot sauce label here. Angelo: Oh yeah, I see that on the one I have. I was going to compliment you on the art too, but now I know that was AI generated. That’s great! Shekeib: But yeah, it's helped me out because before that, when I was making logos for my business, I had to contact so many graphic designers and it cost me money, time. It was a whole month long process to make just a logo for my consulting business. But now I can make labels for hot sauce. I can even make them in the, you know, impressionist style and make it look like Van Gogh did it or something. It’s crazy. Angelo: Yeah. It's phenomenal. Okay, so what's next business wise? I know you both have your career paths but you're making this hot sauce yourselves. It's not factory made. Are you planning on continuing that? Growing it to see where it goes or what's your thought? Shekeib: Personally, I want to just keep going and enjoying it because of how much I'm learning and not to expect too much out of it or, you know, put my life savings on or anything like that. But I think the real potential of it is an education kind of thing where we can learn about AI through the hot sauce. That's what I'm most interested in doing. And if I do mass produce it, I want that to be the core idea of the sauce is now that you learn how we can do this, maybe educate them more about what AI can do on a broader scale or how it can be used for them. My main goal, my vision is to educate people about AI. So if it does get produced at a bigger scale, that would be the main goal. Shohaib: I would be interested in seeing like how it would do in creating, like, because Tabasco say, has only three ingredients. I know it's aged for a while but how would it do with those three ingredients? And I think that would be interesting to maybe compare and contrast the two, so I thought that low hanging fruit on hot sauces, I thought that would be pretty interesting. Angelo: Yeah. Well, you know, best of luck to you, first of all. Náture our producer has tried the hot sauces and she says they're awesome. And I said, don't tell anymore. Don't tell me. I don't want to be able to differentiate it. Don't give me any like, spoilers. But, she says it's awesome. So, we'll include the link and your social media and the link to get it in the show notes. And hopefully this will take off and accomplish that goal that you want. And, yeah, we wish you guys all the best. Thank you again for being on the show. I really, I love your background and your story, and I love your passion for what you're doing. I also love, of course, your passion for computer science. And I think it’s shown the practical side of computer science because it's not just about programs that happen under the covers in some cloud. It's about things that touch our lives. And that's really what this season's about. Shekeib: That's great to hear. Well, thank you for having us. I have been looking forward to it and whenever I found out we were going to be on it I was just so excited. So I want to thank your producers as well for finding us in the first place. It just meant so much to me that people are actually interested in something like this, something that I have a passion for, it just makes it feel so much more important and like other people are enjoying it too. It makes me just want to work so hard at it. So thank you for that. Shohaib: Yes. And it's people like you that, that go into computer science and, you know, make a big, you know, you creating these podcasts it does really motivate me and us to kind of go further. So, hearing about your background has given me a lot of motivation as well. Angelo: Well, likewise. I'm your host, Angelo Kastroulis and this has been Counting Sand. I want to take a minute to thank our listeners. We really appreciate you following us on the podcast and we appreciate your interest. This one was a little bit different and interesting because we got to bring together a human side of computer science. And I think that that's what this season is going to be about. Next episode, I don't normally do this, but the next episode, we're going to taste test the hot sauce, my partner and I, Petter Graff, compare it with some other hot sauces, see if we can figure out which one was AI-optimized, so you don't want to miss that. If you haven't already, please take a second to follow us and to connect with us on social media. You can follow us on Twitter, my company is @BallistaGroup or @AngeloKastr, me personally, or you can find us on LinkedIn @AngeloK1. And you'll find all of our social media contacts including that of our guests down below and if you want to buy the hot sauce, we'll give you a link to that too. Thanks for listening.