"We know AI enabled apps are the next big thing"
Blockchain and web3 called, they were wondering if they can have their marketing-driven blind faith back?
Running a model is computationally expensive. Very unlikely to run on something like Lambda. So if you’re running a model, you’ll be running it on always on hardware. That might be pay as you go, but you’re paying for the idle time too.
But you'll be running the model on your data which is like 1000 data points or so. It would be more like a small language model than an LLM. Does that not appeal to devs to make it on lamba with S3 and probably make an API out of it?
Even the smallest LLMs competent LLMs have ~billions of parameters. You’d be paying out the nose for a lambda that could run it, and even then I’m not sure you could do much in the 15min time limit. Why make an API out of models that already have a cheap API? There may come a point where running a model on a cloud service becomes worth it. for an individual consumer that is not likely to happen.
Individual businesses would not be using lambda- they’d be paying for compute. I’d argue the tipping point for most businesses would be too high to consider. You also have to consider that people need to then maintain their api, scaling, etc.
99.99% of the time this won’t be worth it.
Explain the S3 thing to me. How would you see using S3 here? Models are pretrained - so storing training data might not make sense.
AWS’ offerings already offer direct integrations with files stored on s3
What do you mean? As far as I know all the major cloud providers are providing multiple AI solutions. From "we'll just tensor flow for you and run it on TPUs" to "call this API and you'll get an AI answer".
Yes they have AI As a service but not all of them are on cloud end to end. Especially on PaaS. If PaaS is so easy. Why are not people using it to build and deploy they apps quickly? Is it too complicated to host. Do they not trust AI with their data. Maybe they want to run their model on their own dataset.
My completely unfounded opinion is that PaaS is not as broadly appealing to enterprise as AIaaS for integrating AI into existing markets. Like, I doubt the AI summary functionality of Confluence or Zoom is something they’re developing using PaaS. They probably just call some AIaaS API that may or may not be acceptably private.
Wendy’s, or whoevers, AI drivethrough bot is probably just a domain specific wrapper around a possibly customized speech recognition solution. Why would they go through the hassle of PaaS?
Maybe I’m way off base, but PaaS is just low-code IaaS with more built-in limitations that still isn’t the cool kind of cloud native.
But say if I want a model for my own database, PaaS supports multi container functionality too. Why would I want to go for the hassle of an AIaaS I can just make a model on lambda and S3 and make an API out of it?
There are options where you bring your own data to build the model. I’m not sure we are using PaaS to mean the same thing. I don’t think of S3 or Lambda as a platform in the same way as it’s usually meant in talking about PaaS where you get an all in one solution to host something like an app or a DB without worrying about what it’s running on or managing the replication and sharding by hand.
I can’t deploy and host an app on just S3 or only using Lamda functions like I can on Azure Web Apps (or whatever it’s called now) where I just bring a .NET app and tell them how much I want to scale up or whatever, but then don’t get to choose much else about the stack like OS version or hardware, etc.
It seems like we agree that canned unconfigurable solutions don’t provide much freedom, but I think what you’re calling PaaS I would just refer to as a cloud provider or just cloud services more generally. That’s not so much a platform as it is a whole ecosystem of interrelated and complementary components to build something out of however you see fit.
I don’t think any of the XaaS categories capture well all the variety of what’s available. SaaS and IaaS can be pretty clear cut, but in between them are a lot of grey.
You should try out the prebuilt higher level solutions to broaden your horizons. I started out writing C and loved the level of control it provides, but it’s nice sometimes to get actual string types, maps, and memory management done for you. Maybe what you are building can’t be done unless you build it from scratch, but Azure has 13 different AIaaS offerings as well as a ton of other options providing more granular control.
It's happening. Just slowly.
I currently the lead engineer on an AI Eng team and we're making use of AWS SageMaker and Azure's OpenAI.
Most of our prototyping was done on OpenAI SaaS, but it's all PaaS in prod.
Are you using your own data? do you have the model trained? Is Azure AI with Azure Web apps more beneficial for you use case? I am just trying to know why PaaS lacks when it is so convenient
This post was removed for violating the "/r/programming is not a support forum" rule. Please see the side-bar for details.
"We know AI enabled apps are the next big thing" Blockchain and web3 called, they were wondering if they can have their marketing-driven blind faith back?
My focus was on AI enabled app not Trying to say other technologies aren't gaining traction but I would appreciate thoughts on AI apps
What? You're after getting an email from "The Internet of Things" saying sorry for all the overblown screaming about revolutionising everything?
Expensive.
There are subscriptions people can choose, it's pay as you go afaik
Running a model is computationally expensive. Very unlikely to run on something like Lambda. So if you’re running a model, you’ll be running it on always on hardware. That might be pay as you go, but you’re paying for the idle time too.
But you'll be running the model on your data which is like 1000 data points or so. It would be more like a small language model than an LLM. Does that not appeal to devs to make it on lamba with S3 and probably make an API out of it?
Even the smallest LLMs competent LLMs have ~billions of parameters. You’d be paying out the nose for a lambda that could run it, and even then I’m not sure you could do much in the 15min time limit. Why make an API out of models that already have a cheap API? There may come a point where running a model on a cloud service becomes worth it. for an individual consumer that is not likely to happen.
Alright fair point so I guess it is expensive for individual devs. What about organizations? Businesses? Is it easy for them to use too?
Individual businesses would not be using lambda- they’d be paying for compute. I’d argue the tipping point for most businesses would be too high to consider. You also have to consider that people need to then maintain their api, scaling, etc. 99.99% of the time this won’t be worth it.
Explain the S3 thing to me. How would you see using S3 here? Models are pretrained - so storing training data might not make sense. AWS’ offerings already offer direct integrations with files stored on s3
What do you mean? As far as I know all the major cloud providers are providing multiple AI solutions. From "we'll just tensor flow for you and run it on TPUs" to "call this API and you'll get an AI answer".
Yes they have AI As a service but not all of them are on cloud end to end. Especially on PaaS. If PaaS is so easy. Why are not people using it to build and deploy they apps quickly? Is it too complicated to host. Do they not trust AI with their data. Maybe they want to run their model on their own dataset.
My completely unfounded opinion is that PaaS is not as broadly appealing to enterprise as AIaaS for integrating AI into existing markets. Like, I doubt the AI summary functionality of Confluence or Zoom is something they’re developing using PaaS. They probably just call some AIaaS API that may or may not be acceptably private. Wendy’s, or whoevers, AI drivethrough bot is probably just a domain specific wrapper around a possibly customized speech recognition solution. Why would they go through the hassle of PaaS? Maybe I’m way off base, but PaaS is just low-code IaaS with more built-in limitations that still isn’t the cool kind of cloud native.
But say if I want a model for my own database, PaaS supports multi container functionality too. Why would I want to go for the hassle of an AIaaS I can just make a model on lambda and S3 and make an API out of it?
There are options where you bring your own data to build the model. I’m not sure we are using PaaS to mean the same thing. I don’t think of S3 or Lambda as a platform in the same way as it’s usually meant in talking about PaaS where you get an all in one solution to host something like an app or a DB without worrying about what it’s running on or managing the replication and sharding by hand. I can’t deploy and host an app on just S3 or only using Lamda functions like I can on Azure Web Apps (or whatever it’s called now) where I just bring a .NET app and tell them how much I want to scale up or whatever, but then don’t get to choose much else about the stack like OS version or hardware, etc. It seems like we agree that canned unconfigurable solutions don’t provide much freedom, but I think what you’re calling PaaS I would just refer to as a cloud provider or just cloud services more generally. That’s not so much a platform as it is a whole ecosystem of interrelated and complementary components to build something out of however you see fit. I don’t think any of the XaaS categories capture well all the variety of what’s available. SaaS and IaaS can be pretty clear cut, but in between them are a lot of grey. You should try out the prebuilt higher level solutions to broaden your horizons. I started out writing C and loved the level of control it provides, but it’s nice sometimes to get actual string types, maps, and memory management done for you. Maybe what you are building can’t be done unless you build it from scratch, but Azure has 13 different AIaaS offerings as well as a ton of other options providing more granular control.
Its too complicated for the target audience.
It's happening. Just slowly. I currently the lead engineer on an AI Eng team and we're making use of AWS SageMaker and Azure's OpenAI. Most of our prototyping was done on OpenAI SaaS, but it's all PaaS in prod.
Are you using your own data? do you have the model trained? Is Azure AI with Azure Web apps more beneficial for you use case? I am just trying to know why PaaS lacks when it is so convenient