Simplifying Computer Vision AI for Business - Intel® Geti™ Platform

Tony [00:00:04] Welcome to Code Together, a podcast for developers by developers, where we discuss technology and trends in industry.  I'm your host Tony Mongkolsmai.

Tony [00:00:17] As AI gets more and more complex, the AI community is building more and more tools to simplify end user business use cases. Intel Geti is one of those platforms. Intel Geti software enables teams to rapidly develop AI models and the intuitive computer vision solution reduces the time and domain expertise needed to build models. Today, we're going to learn a little bit about how Intel Geti could help you build a better AI solution. To that end, we're joined by two engineers from the Intel Geti team. Ashutosh Kumar drives technical marketing efforts for Intel Geti. He holds a Ph.D. in materials science and an MBA and has over ten years of experience in software development, R&D and product across semiconductor software security and machine learning domains. Welcome to the podcast, Ashutosh.

Ashutosh [00:01:01] Thanks, Tony. Thanks for having me.

Tony [00:01:03] Ali. Hamilton is the Intel Geti Technical Product Manager, shaping the roadmap items and future of Intel's computer vision training platform. He has a Ph.D. in computer vision and has been working in the field for over ten years on the use cases from security, medical and agritech. Welcome to the podcast, Ollie.

Ollie [00:01:21] Thanks very much. Glad to be here.

Tony [00:01:23] So Intel Geti is one of Intel's newer AI platforms and we were really excited about announcing it last year at InnovatiON. Ollie, why don't you tell our listeners a little bit about what Intel Geti enables?

Ollie [00:01:38] Sure, so Intel Geti is really your one stop shop for both your data management, your annotation, your training and model optimization as well. It just brings together all those components you need to build computer vision models into a single platform so you don't have to keep going in and out of different things. Importing, exporting all the time. You've got it all at your fingertips just as you need it.

Tony [00:02:00] And what types of models am I able to put in there? So there's there's a variety of computer vision models out there. Intel Geti as a platform, you said, allows me to kind of optimize those models more easily. Which models can I use in there? Is it just any computer vision model I want or does it have to be a specific subset?

Ollie [00:02:19] Yeah. Currently today we've got a range of models for each task type. We cover most tasks that you'd expect today in computer vision. So, you know, your detections, your segmentations, your classifications, anomaly tasks as well. And then within each, each of those broad task categories and you've got different architectures within those to select from.

Tony [00:02:41] Okay. And Ashutosh, which customers are we focused on right now? Is there a specific customer that would really benefit from Geti, or is it really anybody that's looking to use computer vision to enhance their business case?

Ashutosh [00:02:55] Yeah. So Intel Geti is primarily a horizontal platform. Customers from any industries like industrial manufacturing, healthcare, life sciences, smart cities. They can use Intel Geti to build custom vision models for their use cases with their own data.

Tony [00:03:14] So let's let's walk through an example to make it kind of more concrete. You said manufacturing. If I was in manufacturing and I was building, I don't know, a widget, this is back to computer science 101, if I was building a widget, how would I use Geti as part of my manufacturing process?

Ashutosh [00:03:32] Yeah. So there are a few different ways where you can actually use computer vision models so Geti, we'll talk about Geti in a little bit, but let's talk let's talk about how we can use computer vision more in those cases, right? So if you are putting a product production line monitoring camera system, you need some kind of machine vision or computer vision model in those camera systems to be able to provide you, let's say something when your widget is coming defective provide you that information. If you are like putting together a few different widgets to build something else, you need some kind of a quality assurance mechanism there. Yeah. And you want computer vision to be able to assist with that kind of quality assurance. Or if you have a robot that you want to empower with and analyzing different components that's coming together on your production line, that also needs computer vision intelligence. So building those kind of computer vision modules there with your own custom data, that's your property data that's not available in the public space. That's where Geti can help. And I can give you an example, actually. We have a case study on our website with a company called Bravent, and they are based in Spain. They have a large agricultural equipment manufacturer as their client, and they were building a quality assurance system like automated quality assurance system for their client that an entire quality steps required 90 different parts to be assembled within 2 hours by a technician and doing that four times a day. So there is bound to have like you have different types of mistakes happening during when you are doing that kind of repetitive task. Yeah, so they built 20 different computer vision modules in a very short timeframe using Intel Geti and enabled their client and those technicians with an automated system that can help them making sure that they're delivering quality product when they're going to that assembly process.

Tony [00:05:55] Before Intel Geti, if I wasn't used in using Intel Geti, I'm going to just kind of walk through this in my head and see if it if it makes sense. So I would have set up my cameras. I would have captured those images. I would have had to put them in some type of data pipeline, kind of try to extract features from these images and then somehow annotate the data of what looks good, what looks bad. Right. And I'm essentially I'm manually doing all of that either through code or something like that, kind of like setting bounding boxes of where where my, my bad parts are, right where I say, this doesn't look good. And I'd have to do all of that kind of through code and through manual inspection. And there are tools that kind of help me do that. And then after that, then I have to put that in a computer vision model. I'd have to choose the model and have to train it, etc.. Now I'm I'm switching over to Geti. So, Ollie, what does that look like as I'm switching over to Geti? What are the different pieces that Geti provides me that now I don't have to do otherwise?

Ollie [00:06:57] Yeah, I think you've probably glossed over a few steps there as well. I mean, not only do you have to do that, you have to set up your environment. You have to get all your versions correct when you're building the model, all that dependency hell that you have to set up before you even get to those stages that you mentioned. So I mean, that's that's the first stage of Geti. You don't have to worry about any of that. We've done all that hard work for you. Our engineers have toiled tirelessly for hours and hours. I can testify to that, that they've they've made it so you don't have to worry about that environment set up that you can just now get on with the the kind of getting your knowledge into the platform, which is essentially just creating those annotations. Then you don't have to then have that second stage of right. I've got my annotations in this one, tall enough to export it into a different tool, maybe have to do some conversions because I've got different definitions of bounding box coordinates or the polygon coordinates or something. And what we've done is remove all those little barriers that face customers previously or anyone trying to build models and just trying to make that seamless experience so you can just go from uploading your data, creating those annotations just with some simple web interfaces like you would expect with a...you can draw, you can draw boxes, you can draw polygons, and then that's just automatically all collected up for you, then sent over the training backend seamlessly without having to know what's going on. And then it just handles it all for it, does all the training and then provide you with a model and without you having to worry about any lines of code, all these environments that you need to try and configure.

Tony [00:08:28] So then if I'm, for instance, a naive user, if I, if I didn't know how to do all of that, but I was able to at least set things up and capture the images, what you're saying is that Geti actually should allow me to, without really knowing anything other than I've set up Geti. I'm uploading images into the system. I can actually use the interface in the Geti app itself to do all the annotations for all of the images that I've uploaded and then kind of just hit a button and train it. And then I'm going to have get a model out on the other side?

Ollie [00:08:58] Yeah, you don't even have to hit a button by default, but by default, automatic training is turned on. So it just automatically in the magically in the background, change this model for you. And very often when people first get it, get a hold of this, particularly the group VP instantly just gave it to his child and said here you go train that. And I can't know how old they were, but I have had multiple children training these thing, you know, from 6 to 10, training detectors on their toys or on their that dinners, on food and whatnot. And it is it's literally child's play.

Tony [00:09:34] That's that's one of the really cool things about Geti having seen a demo of it a couple of different times. I think that the one of the challenges of AI when I gave a talk about what AI looks like to a business, to a bunch of, you know, like CEO CTOs as they were trying to integrate it into manufacturing earlier this year at an IEEE conference. And I think I showed them there was probably like 12 to 15 people that needed to be involved in creating the AI pipeline, like just different personas. And the really cool thing that I get out of looking at Geti is that I'm I'm actually compressing that. Obviously, you're going to want people in the background who can verify that things look right to them. But probably instead of having 15 people, I probably only need like seven or eight because you're taking out all of these steps and this nuanced expertise that has to exist to create an AI model and simply replacing it with a deployed tool. Right, Right. I mean, because so I guess maybe that would be the next question. Let's talk about how do you get access to Geti? What does that deployment look like? So Ashutosh if I'm if I'm now interested in saying, let me go try Geti out, where would I go look for and how would I gain access to it?

Ashutosh [00:10:52] Yeah. So the platform is becoming generally available pretty soon. So we have just released our latest and greatest 1.5 Release that brings out major enhancements like adding support for ONNX model to be able to use and more advanced users to be able to build cross-platform deployment using ONNX as well. And then we have also added support for FP16 models, FP16 model formats with OpenVINO so that also enables our customers to take advantage of faster compute on Intel dGPUs with of course with higher accuracy because if you're quantized to it, you are definitely losing some accuracy. But like in some use cases, you want to preserve more accuracy, so you would probably go for FP16. So to get access to Geti, we have a web form on the website you can request a...you can fill up your information there, and then our business development team will reach out accordingly based on your needs. And we will also be doing some webinars later in the quarter. And so keep an eye out for those.

Tony [00:12:18] One of the places that is really popular I know this for the podcast that I've had is anytime I do a podcast around health care and AI, so do you guys have any good success stories around using Geti in the health care domain?

Ollie [00:12:35] Yeah, one of our earliest partners is a hospital in the UK in London called the Royal Brompton. They've been using it to look at this disease where I need to get the pronunciation right primary ciliary dyskinesia, and basically the the small hairs that are in your airway that help move mucus around. They are defective in some people who have this disease. And the way they do the diagnostic now is by taking nasal scraping to obtain some of these hairs, and then they take an electron microscopy cross-section of them. So you end up with these lots of little circles on this electron microscopy image, and there's a field of hundreds of these on on the slide and they have to go through manually and look at all these. And within the cross-section of each one of these cilia, there are several different components within the microtubes and all these little arms and things that I can't remember the terminology for. And they have to look at these and analyze them and look for these defects. That's a lot of work to do because they have to analyze I think it's like 300 or so of them. Takes hours and hours and hours and you each have a pretty good eye to to spot these defects. So they they've been using Geti for a while and the actual the predecessor and they've come with us through that journey from the predecessor to Geti and they have trained a series of models that can do that for them so they can now detect where all these cilia are and this large image. Then they can then classify the cilia and look at all the different defects that the system can then pick up. So rather than spending hours looking at these slides and having to manually inspect them, they've got it down to 2 seconds and then that's greatly improved throughput. So they've then been running this in parallel with that with the experts in that group. Also, you know, you don't wanna jump on board with the machine learning stuff too fast when it comes to clinical uses. They have to go through stage at various stage gates when it comes to the of what kind of regulatory approval. And what they've found is that as they've gone through this project and it's evolved with them, they've now reached the point where it's performing on par with the clinical experts that have 15 years experience in the field. So that's given them the confidence that this model has now reached a point where it's it's performing on par with them and even exceeding them in some areas. So that's that's really not only empowered them to reduce the amount of time they spend on that manual work, just counting things and looking at the defects, they can then start to offload this to the system as it goes through the regulatory steps.

Tony [00:15:11] Yeah, that's really exciting. Just because you think about how much, not just how much time they're saving, but you might get better accuracy or on prior accuracy. But then that like you, I think you were mentioning earlier how how much that frees them to have the time to do real work. Right. One thing is just manual annotation. And now, instead of just annotated spending their time annotating and counting, they're probably more likely to spend all that excess brainpower actually trying to solve the problem and understand how to combat the disease rather than just identifying the disease, you know, among a wide swath of patients. That's really cool. So we've we've talked a little bit about how this makes life easier for people who are not potentially experts, which is really cool. If I'm an AI developer, what are the advantages I get out of using Geti?

Ashutosh [00:16:05] So, yeah, that's a very good question, Tony. And the whole idea of Geti is actually to not just empower those domain experts to subject matter experts, to be able to build computer vision models, but to bring the entire team together from the developers to the scientists to the domain experts. So you do have those intuitive user interface that makes it very makes it look very easy to build in your computer vision model. But the power behind look behind all those computers and model training is also accessible to develop developers and data scientists. They can change different architectures. They can modify all these learning parameters, hyper parameters, augmentation parameters, etc. All those things are available through the UI. But we also have the RestAPI. Rest API end points available for developers to build the entire MLOps integrations with. And not only that, like if you're using Python for your development pipeline, we also provide software development kit that provides very easy to use functions around those RestAPIs that you can directly utilize. Build your deployment pipeline, run experiments in the platform if you don't want to interact with the UI directly. So you can you can do those in an automated manner.

Tony [00:17:40] So it can kind of fit into your MLOps platform as well.

Ashutosh [00:17:44] So yes, you can integrate into your MLOps platform for for your full pipeline. Yes.

Ollie [00:17:49] Yeah. I think from my perspective, it's I like to say it's doing more with less, with less being your time as a data scientist, as a machine learning expert. You only have some amount of time in the day that you can do these things. So a lot of the time you're doing a lot of boilerplate work. You've got your scripts for cleaning data, for for converting annotation formats or whatever. If we can get rid of that part of that boilerplate, then you can focus on solving more problems. I myself from a computer vision backgrounds and I use Geti myself quite a lot myself for...I've got little projects around the house, some for nature monitoring, things like that. And it means I can just spin up a project over the weekend and I don't have to worry about, Hang on, This is actually quite a big body of work that I need to worry about with my dataset cleansing and annotation conversion. I've just got a problem that I want to solve quickly. I can just do that drop down onto the next thing and means, as I said, I can do more with less of my time essentially.

Tony [00:18:50] Yeah, that's a really good point because one of the biggest challenges I think with AI is it everybody thinks it's life changing, but we're all still playing with simple demos that somebody else has set up. And what I really want to do is figure out how do I get it to solve my problem quickly without having to understand all of the different pieces. I literally just want it to solve what I needed to solve. And I don't want to have to figure out how do I set up a cluster if I need to do some scale training or how do I set it up on my system and get all the frameworks installed correctly. I literally just want to push a button and get something out the door and Geti is getting you that at least for computer vision right now. I know I've talked to you guys a little bit in the past about potentially future paths. So obviously people are talking a lot about large language models, the ChatGPTs of the world, the use cases of text to something in the world with its text to image. Text to text, text to speech. Is that something that the Geti team is looking at, also trying to simplify in the future?

Ollie [00:19:55] Yeah. I mean, we're definitely investigating that, saying how all these multimodal or large language models can be utilized well, not only from from the actual platform itself, but even from documents. No, no one reads documentation, apoligies to our documentation team. But, so how can we make that easier? How can you make traverse and documentation easier? Can we have an interface that allows people to just ask questions, for example, and in get that guidance through the platform rather than needing to troll through traditional documentation? Then on the machine learning side, how can we integrate these large language models into the vision side as well? So that kind of multi-modal approach to two things is, one, it's one thing to be able to train a detector to find an object, but then if you've got that extra understanding that we've seen in some of the GPT-4 models, where then you can start interrogators to ask what that state of the object is without necessarily having to classify every aspect of it. So if we can start integrating these these more advanced components in the future, which the team is looking at, I think, yeah, the potential is just going to skyrocket.

Tony [00:21:04] So what would that use case look like? So with the with Geti, as it is right now, where you're kind of it sounds like you we annotate data, we help identify areas of interest within a computer vision model through our interface. How would the addition of a large language model to that... What would be different in this case? Because right now it's kind of a you have like an image stream or video stream, and then you're coming in. Then you're saying, okay, within this new image, I can identify the anomaly based on what I've previously seen in the past. What's the addition of a of a large language model give me there? What different use case would I have potentially if I did that type of integration?

Ollie [00:21:44] So one I tackled recently at the weekend. Again, another one of those little projects that I was training a detector to monitor my birdfeeder and I wanted to see how much level of seeders before need to go and refill it. And I can solve that pretty easily to take a few hours to train that model, just to train a bounding box detector to look at the level of seed. But I had to take a bit of manual effort. Still, I had to annotate different amounts of seed within the within the feeder so I could learn to recognize that I can see what happening in the future is I can just train the detector on the bird feeder and then at some point I can say, describe how much seed is in this feeder. Once those models start having a bit more knowledge of the world around them, then I don't have to worry about annotating for the different states and the condition of it. Then I can say just notify me when it gets a 10% capacity. Once they have that that concept of some of the physical world which we are starting to see in some of the some of these models, then that ability to just say, here's the object I want, tell me when it gets down to 10% and then send me a text or something.

Tony [00:22:50] It sounds like you've created a new side business for yourself.

Ollie [00:22:54] I've got a few.

Ashutosh [00:22:55] I did plug in that big box by Twitter account in in the chat.

Tony [00:23:02] Oh, awesome. So we can actually we can put a link up there and people can see how much how much proceeds left for your birds.

Ollie [00:23:09] So the the one in the chat is the first iteration of that which is monitoring the bird box. So that was monitoring when the birds were there, when they lay their eggs and, and monitoring their visit for frequency. But now that they've left, that breeding season is over. I was actually left with my camera idle. It wasn't being used for that anymore. So now I just pointed at the bird feeder instead and I retasked to something else. I think that's that shows the incredible flexibility of visual models. And therefore the one instance is during bird frequency visits, monitoring natural bird itself. And then another instance, the exact same sensor has been used for a completely different use case. It's about seed level and that ability to just retasked sensor to a different use case so easily and so quickly. I don't think we've seen that before in technology as a whole.

Tony [00:23:59] Yeah, that's very cool. So for developers who are listening, who want to invest in Ollie's new new bird box bot feel free to reach out to him. Speaking of startups, so one of the things I did want to talk about was how the Intel Geti product came to be. And I don't know when you guys ended up joining the Geti team, but I know that it was it a project or a company that was in the Intel incubation system, right? Where each of you guys part of that before they were acquired by Intel.

Ollie [00:24:34] So I was actually the the first employee of the company called COSMONiO and founder and CEO Yannis Katramados, who now leads the the Geti Group. Yes. I mean, we started we were acquired about three years ago and in the company we started about ten years ago. So and now we've been building this product for quite a while. That's the original incarnation of it. So it's been several years of blood, sweat and tears building it to the foundation that then when we came into Intel about three years ago, that then we could build on top of that and carry it through to where it is today.

Tony [00:25:08] Okay, so were you guys part of were you actually part of an Intel program or were you just a company that was acquired by Intel?

Ollie [00:25:13] Just a company that was acquired, Yeah.

Tony [00:25:16] Okay. All right. And so I I'm going to digress then a little bit just because I can. So I've been talking to a lot of people who are in the startups program for for Intel and for oneAPI. And I'm I like to talk to developers about their experience being part of a startup because I think a lot of developers who are listening are always interested in that. When do I know that I should go work in a startup versus what I was doing? So how did you decide that you wanted to go work for the startup versus potentially taking a different job in computer vision at you know, one of the standard big companies that was working on this like a Microsoft or a Google or something like that?

Ollie [00:25:59] Yeah, I mean, my route is perhaps a little bit different. So just as I started, my PhD, Yannis was finishing his and he started the company at the same university. And I decided that, hey, I don't have enough pressure on with a PhD, so let's, let's go and join this as well and do this part time work. And initially the company was a consultancy, so computer vision consultancy work going from project to project to project, which is fantastic, especially in the early days, you get to see all these different use cases coming up. But that kind of drove the need for this, this product to be created. Every time we were doing a different consultancy project, we almost had to become the domain expert as we worked with our customer and said, Oh, we've got this problem to solve. We have to understand that problem quite in-depth so we could build the models to solve it. And that meant now as you go from agritech projects to security projects, it's a huge effort on the developer too, to learn each of those domains. So rather than just rehashing the same code again and then having to learn the different domains, we thought, well, let's just build this one code base which turned into the product and then give it to them. The people who know what they're talking about in their domains and let them do the work really.

Tony [00:27:16] Yeah. So building a solution for the problem. Makes sense, where all good, good businesses come from.

Ashutosh [00:27:22] I come to computer vision from domain expert side. I am a material scientist by training.  All those on this team, which is that domain experts would probably be using and not not wanting to spend hours just analyzing by themselves, rather have an AI module to help find him.

Tony [00:27:44] So you're really happy that that computer vision, AI and these these products have come along to make your life easier... So since this is a developer podcast, I'm sure a lot of our developers would be interested in what getting looks like in terms of like software stack, what software components actually go into geti. Can one of you guys describe the different pieces of software that Geti actually integrates, so ideally, developers don't have to?

Ashutosh [00:28:13] That's a very detailed question and probably we can talk hours to answer that. But yeah, just to keep in mind, like for, for a data science ecosystem. What you need, basic things starts with Python and then you have different types of frameworks that you start working with, right? And then building your models. Then you haveou have something which really basically you have one system which is basically managing your annotations data annotation state that gets fed into your training pipeline. You're building your training pipelines in Python and then you had if you have your optimization pipeline. So you want to integrate those pieces as well. If you are writing some kind of inference, like during, let's say, if you're also connecting your internal serving pipeline into them, so you want to build all those connector pieces together. So from software stack point of view, like all these components are like attribute annotations, dataset management service is like training service or inference service. All these things are it's a microservices based architecture that is orchestrated using Kubernetes, and each service is doing its own job and communicating with other pieces in the entire services stack. In most of the modules that we are supporting today, actually all the modules that we are supporting today are by PyTorch framework based. But the way we have designed the architecture is like in future, if we have models on other frameworks that we will be able to bring those models in in the design and architecture as well. And optimization for optimizations, we are we are very tightly integrated with OpenVINO, so when you are building your models, you get OpenVINO optimized modules out of the box. You don't have to build your own optimization pipelines. For annotation management as well we have a lot of different email capabilities, especially. One of the important thing that we have is that Intel's own data asset management framework, that that is also called Datumaro among the developer community that helps. Like if you have datasets annotated in, let's say, more popular Cocoa, that obviously you're looking at formats and targets that Datamaro framework enables make on conversion into the formats that we are reading in the in the platform. So these are some of the flavors. And as I mentioned, we can we can probably talk hours if you go to deep into the different elements of that of the platform.

Tony [00:31:16] So we walked through the some of the the examples of Geti that I understand, which is kind of annotating data and trying to identify either anomalies or current states of of something through computer vision. Are there any pieces of Geti that I'm missing? Are there any cool pieces of technology that we have not touched on yet?

Ollie [00:31:36] Yeah. So one of the one of the cool components is something called active learning. And this is by this process whereby the the user works with the model and the predictions from it. And then the system all suggests the user which of the optimal images to annotate. Right. So if you were to build a dataset out, you might just naively go through every frame and annotate them. But often if it's a video, there's little value in annotating consecutive frames because that one's similar to the previous one and similar to the next one. You can also randomly select images throughout the video, for example, or images throughout your dataset, but that's not, again, the most optimal, optimal way that these models train. So if you have a process by which the the system will suggest to the user which ones are the most optimal frames to annotate, it's the whole notion of you get better at practicing things you're not very good at. Rather than keep doing the things you already know how to do. So that's what the system does. It suggests to the user, Hey, I think I'll be better off learning from this frame. Can you tell me if I've got the prediction correct or not? And then you feed feedback to the system? Yeah, you did a good job. You got it right. Or maybe you need to take it a little bit, update the annotation feedback for a second round of training and then the system will learn quicker from those optimize data sets.

Tony [00:32:55] That's actually a really cool optimization too. So instead of you going through and just you as the user going through and trying to annotate the things that you think are important, the system itself actually will say, I think I understand these frames, so therefore I won't show them to you. But here's the frame I'm not sure about. So help me out here and provide your user input to make sure that I am training on the right things in the in these frames.

Ollie [00:33:19] Yeah, exactly. It's all about and again that that more with with less. How can we use as few data as possible and and try and get to that same level of performance with the model as fast as we can with as few of your data as possible.

Ashutosh [00:33:33] I do want to add one more thing that active learning algorithm help, right? If you have like variations in your sample, in your data sample take and this kind of human in the loop approach with active learning algorithms that enables those algorithms to fit data sample from like across, you get variations in the sample itself so that it eventually codes reduce sampling biases where we have, like even our Intel Labs team has published some research and like the algorithm active learning algorithms that they have developed, how it helps reduce the sampling bias. So that's scientifically proven and that actually helps in real life use cases with our data customers as well.

Tony [00:34:20] So we're almost out of time. And so I'll and like I always do, by asking each of you, where do you hope technology is going in the next couple of years? And sometime that's that you could answer that in a totally generic space. You could answer it around Geti, the Geti platform itself, wherever you guys want to take it.

Ollie [00:34:40] Wow. That's a big one. I suppose there's two ways of looking at this, where I think it's going and where I hope it's going. Where I think it's going is, that's pretty difficult too to predict. Looking at the advances in the last couple of years, we've seen with the the generative AI or the the image generation stuff and the ChatGPTs coming out. I don't think I could have seen those coming and the rate at which they've evolved as well. That's been phenomenal. Obviously, we're going to see those continue to get to get better, that the quality is going to improve the of the image generation stuff, the the quality of the predictions coming from the large language models. And I think they're going to converge as we're starting to see with them now anyway. Where I hope it goes is being able to achieve this kind of stuff with lower compute power. We see huge amounts of resources being being devoted to training these models. And to an extent, they're they're fun. They're really cool what they do. But the utility. Is sometimes a little bit in debates might be a bit controversial, but now that there there's a limited amount that society can gain from generating a new fantasy scene of Middle Earth or something, whatever, And the amount of energy put into that has been absolutely phenomenal. And the amount that continues to be put into generating them, running them inferencing on day to day is is quite scary. So I think obviously we need to continue progressing these models, keep feeding them with more accurate and more relevant information, but also hopefully reducing that that power draw and particular on inference as well, if we can bring that down. Training's done once or twice now. You don't tend to improve these things a few times, but then when you deploy it, you've got millions of people trying to consume this thing. So actually you end up using far more power at runtime when you've got millions of endpoints trying to consume this than you do necessarily at training time. So when we start deploying these things out to millions, potentially even billions of people in the future, getting that that power consumption down, that essentially CO2 footprint as well in the future and getting that low, I think is where I hope we see things going.

Ashutosh [00:37:02] Yeah, I couldn't agree more with Ollie here. I mean, we see so many companies trying to build applications on these large language models that we have seen exploding in six months or so. But we also see that churn rate, like you have those language modules from 2 billion parameters to 500 billion parameters. I mean, those 500 billion parameter modules, if you have tens of millions of dollars to train those models, not everybody has got those, right? and then running those modules at at the compute endpoints at end user locations. It's also it also requires a humongous amount of compute and solving those, giving those interference results in real time also becomes challenging based on the kind of compute that you have. Mm hmm. And that's a client or edge use cases, right? A lot of these that if you if you specifically look in the computer vision space. Right. Like lot of these endpoints, those street cameras or cameras in your production... Like cameras monitoring your production line. They need real time analysis. And you can't wait for all those big heavy models to run for a few seconds and then give you data and give you results. So, yeah, I would hope that there is a convergence. At some point we find that sweet spot of having the capabilities of founation model merging in those kind of use cases that will actually start using results in a more functional manner.

Tony [00:38:47] All right. Well, thank you very much for joining us guys.

Ollie [00:38:50] Thanks very much.

Ashutosh [00:38:51] To you for having us.

Tony [00:38:53] And thank you, our listener, for listening. Join us next time when we talk more technology and trends in industry.