Training takes more resources.
Querying takes less resources.
are there any studies or data on the actual consumption of electricity that AI uses? I know that training consumes a lot, but if the usage of it doesn’t consume much, it doesn’t really matter since it’s a one time cost.
I don’t know of any studies unfortunately, but I did want to point out that training is not quite a one time cost in practice, because training has already been done loads times and is still being done! I’m theory, if we stopped training all AI and just kept the ones we have, then indeed the training cost would be bounded just like you say, I’m just afraid we’re quite far from that.
this is true, but this is always going to lead to a different AI and a different product eventually, it’s not like it’s going to be sustained entirely by the whims of chatgpt 3.5 for example.
FWIW I’ve generated ai images using a solar panel the size of an iPhone as the power source.
The training was the cost.
The data was connected by wire but powered externally, which I argued was cheating but I guess not idk I just work here
anything outside of the device doing the work is an externality, so we can write that off anyway.
Consuming memes on le internets would be consuming whatever that consumes, and people are fine with that, just not AI, so it’s irrelevant anyway.
It would be on the order of aN intensive video game, maybe. Depends on the size of the model, etc.
Training is definitely expensive but you are right in that it’s a one-time cost.
Overall, the challenge is that it’s very inefficient. To use a machine learning algorithm to do something that could be implemented deductively is not ideal (On the other hand, if it saves human effort…)
To a degree, trained models can also be retrained on newer data (eg freezing layers, LoRa, GaLore, Hypernetworks etc). Also newer data can be injected into a prompt to make sure that the responses are aligned with newer versions of software, for example.
The electricity consumption is a concern, but it’s probably not going to be the end of the world.
the one place you would use an LLM is going to be something where nothing else can be used, cataloging information for example.
This pretty much lines up with my understanding of AI.
They can also be really good for quickly writing code if you line up a whole bunch of tests and line up all the types and then copy and paste that a few times, maybe with a macro in Vim.
The LLM will fill in the middle correctly, like 90% of the time. Compare it in git, make sure the tests pass, and then that’s an extra 20 minutes I get to spend with my wife and kids.
And then the one fucking hallucination you missed ends up crashing the whole thing on the weekend.
On the other hand, you could have done that by accident yourself.
Yeah, that’s always a risk, but as you said, humans make mistakes too. And if you change your approach to software development by writing more tests and using strict interfaces or type annotations, etc., it is pretty reliable and definitely saves time.
that’s definitely one of the potentials, if you prefer a schizoprehnic person writing code which you then have to make work. I suppose.
I’m autistic about things and would write it all myself to my very high standards of “how autistic is it? If the answer is yes, than good enough.”
if other people look at the shit im doing and say “you could do this better” it’s not done well enough.
Which I can totally understand, but I would like to spend more time with my family and less time writing code.
This also allows me to iterate faster and identify useful ideas That justify deeper effort.
This is such a stupid take, the author is either a lemming or doesn’t understand how resources and commodities work
If you don’t explain why you hold this opinion your comment is wasted.
A small city usually requires one gigawatt continuous power. 10 minutes of power is equal to 100k kWh.
A 200x200 GenAI image takes about 2 seconds on an RTX 3060 running at 170W. That 2 seconds of power is equal to 0.000094 kWh.
The meme is off by a factor of 1,063,829,787.
So how many images would need to be generated at once to take down a city 👀
exactly 1,063,829,787, can’t you read?
1,063,829,787
It would be that many times the squared volume of a 200x200 image. Which is already 40,000 pixels. Multiply those together and you get 4.25 million megapixels (which are 1 billion pixels each), aka 4.25 gigapixels. It’s extremely large but not unachievable.
People are AI generating entire movies which are 24-30 pictures per second. Seems like you could hit really big numbers really really fast.
People are AI generating entire movies which are 24-30 pictures per second.
I don’t think they’re generating those movies in real time.
because people just make shit up and post it on the internet, and other people content with making shit up and posting it on the internet go “wow good post”
and then nobody bothers fact checking anything ever.
like i understand the anti AI sentiment, but please do yourself a favor and use actually good arguments against it, not stupid arguments that make you look incompetent and silly.
Also it’s not a small city, merely a village. Not even a very nice one.
Not a village. My gaming PC can generate a 200 x 200 avatar in seconds
That being the case, let’s have a couple of avatars.
click ‘no’.
reconfigure.
set dimensions: 50000x50000
DO IT
EMP deployment successful
So, the internet version of the trolly choice problem?
well no, it’s worse actually.
“The name’s Plissken.”
*BEEP*
Do I get to choose the city?
If we all collaborate we can shut down Florida at all times.
I would object, but I’m on vacation right now. It won’t affect me for a week. Generate as much as you can!
Training the models is very resource intensive, but after that it’s basically fine. I can run Midjourney image generation on my modest gaming laptop. Takes maybe 30 seconds for a high-resolution image at a laptop’s “gaming mode” power consumption.
AI image generation is comparable in electric use to e.g. playing Valheim or Elden Ring.
It’s when it’s scaled up industrially that it becomes a huge waste of electricity. Me playing Valhelm for 5 minutes is nothing. My startup creating 20000 bots playing Valheim 24/7 is a problem for society.
It’s the mining of diamonds that kills all the children. After the diamond is mined, I can use it with almost no child deaths. Diamonds are fine.
No, using an already-trained model doesn’t “use up” the model in exactly the same way that pirating a movie doesn’t steal anything from Hollywood.
Use a diamond doesn’t “use up” that diamond.
And yet, it’s still unethical to buy already mined blood diamonds from people who continue to mine more blood diamonds. Funny thing about that, huh
In this analogy, using the diamond does use it up. In the sense that none else can use that diamond concurrently. If someone else wants a diamond, more children must die.
This is different from the trained AI model, which can concurrently be used by everyone at the same time, at very little extra cost.
Even if the diamond mine owners stop mining, it’s unethical to buy their stockpile of blood diamonds.
Also, there is a cost besides electricity - the theft of artist’s work is inherent to the use of the model, not just in the training. The artist is not being compensated whenever an AI generates art in their style, and they may in fact lose their job or have their compensation reduced due to artificial supply.
Finally, this is an analogy, it’s not perfect. Picking apart incidental parts of the analogy doesn’t really prove anything. Use an analogy to explain a problem, but don’t pick apart an analogy as though you’re picking apart the problem.
and they may in fact lose their job or have their compensation reduced due to artificial supply.
highly doubt. Any artists that do lose their job are probably mostly ok with it anyway, since it’s most likely going to be graphical drivel anyway. In fields like media theres a different argument to be made, but even then it’s iffy sometimes. Also i don’t think this would be considered artificial supply, it would be artificially insisted demand instead no? Or perhaps an inelastic demand side expectation.
Although, it would be nice to have some actual concrete data on artists and job prospects in relation to AI. Unfortunately it’s probably too early to tell right now, since we’re just out of the Luddite reactionary phase, who knows.
Any artists that do lose their job are probably mostly ok with it anyway, since it’s most likely going to be graphical drivel anyway.
Replace “artist” and “graphical”, and you just described most jobs. I don’t think most people are ok losing their jobs even if those jobs aren’t especially inherently rewarding; they’re getting paid for their area of training. They’re not just gonna be able to find a new job because in this hypothetical, the demand for it (a living human doing the work) is gone.
I consider this an increase in supply because it’s an increase in the potential supply. Productivity increases (which is what this is) mean you can make more, which drives down the price, which means that artists get paid less (or just get replaced).
Remember: if you 10x the productivity of an employee, that typically doesn’t mean you produce 10x the product, it typically means you need 1/10th the employees. That payroll saving goes right into the pockets of execs.
Also wrt luddites, they weren’t wrong. It did absolutely demolish their industry and devastate the workers. It’s just that the textile industry was only a small part of the economy, and there were other industries who could absorb the displaced workers after they got retrained.
LLMs threaten almost every industry, so there is a greater worker impact and fewer places for displaced workers to go. Also now workers are responsible for bearing the costs of their own retraining, unlike back in the day of the luddites.
A very large amount of those dug up diamonds end up as “industrial diamonds.” Because they are far from gemstone quality. And they definitely get used up. I have used up my share of them as cutting tools when I was a toolmaker.
Ok cool, but this is an analogy. Why are you defending the use of AI by megacorps by objecting to irrelevant parts of an analogy on technicality?
It’s a bad analogy and just plain wrong fact. Do better.
You’re insufferable.
I know the analogy isn’t a perfect fit for LLMs in general. Analogies never come close to describing the entire thing they’re analogs of, they don’t need to.
It doesn’t matter because this is a suitable analogy for the argument. This is how analogies work.
The argument is that because the harm has already been done, it’s fine to use LLMs.
That same argument can be made for blood diamonds, and it’s untrue for exactly the same reason:
Because buying the use of LLMs (which is mostly how they’re used, even if you pay in data instead of money) is funding the continued harmful actions of the producer.I can’t believe I have to explain how analogies are used to a grown ass adult.
this is actually a really debatable argument. If you’re buying it first hand, from somebody trying to make money, yes it could arguably be unethical, but if you’re buying it second hand, i.e. someone who just doesn’t want it anymore, you could make the argument that it’s an ethically net positive transaction. Since they no longer want the diamond, and wish to have money instead, and you wish to have the diamond, and less money. Everybody wins in 2nd hand deals, weirdly enough.
Setting aside “there is no ethical consumption under capitalism”, which is a debatable for another time:
I don’t totally agree with your assessment of 2nd hand sales: it’s not ethical positive, at best it’s ethically neutral, because demand trickles up the market. I could go into this more, but ultimately it it’s irrelevant:
The 2nd hand LLM market doesn’t work like that because LLMs are sold as a service. The LLM producers take a cut from all LLM resellers.
You could make a case that self hosting a free open source LLM like OLlama is ok, but that’s not how most LLMs are distributed.
Setting aside “there is no ethical consumption under capitalism”, which is a debatable for another time:
the simple answer is yes and no, there is both ethical, and unethical consumption under capitalism. As there is in any society throughout human history.
I don’t totally agree with your assessment of 2nd hand sales: it’s not ethical positive, at best it’s ethically neutral, because demand trickles up the market. I could go into this more, but ultimately it it’s irrelevant:
if we start from a basis of ethical neutrality, ignoring the parties as a baseline, i included the parties because i would consider two parties who wish to do business, successfully doing business, to be an ethically positive thing, as two people have had their needs/wants more closely met by this deal. Therefore making it ethically positive.
If you’re starting from an ethically negative point, you need some sort of inherent negative value to exist within that relationship. Perhaps the guy who is selling was hitler for example, that would make it an ethically negative scenario. Maybe after he sold it, he would’ve donated the money or given it to someone who can do something more useful with it, making it even more ethically positive.
there’s an argument to be made for something with perceived value and static supply to have an upwards trajectory in the market going forward, for something like blood diamonds this is unlikely, but assuming it did happen, this should actually be an ethically positive thing assuming that the families of the original diamonds ended up getting their hands on them for example. If they weren’t given back to the family then it doesn’t really matter since you’re back to the baseline anyway. Prospective investments aren’t a real tangible asset, so we can’t forsee that.
although to be clear, i wasn’t talking about LLMs, this is a much harder thing to do with LLMs, although the argument here is that the training has already been done, the power has already been consumed, you can’t unconsume that power, so therefore whatever potential future consumption that could happen, is based off of the existing consumption already. Unless of course you did more training in the future, but that’s a different story. Just boycott AI or something at that point lol.
In both your examples, you seem to assume that the harm is already done and that there is no continued harm.
But in both cases the harm isn’t finished; the blood diamond mine owners use the continued sale of blood diamonds to fund their continued mining operations, and LLM providers use the sale of LLM use to fund the continued training of new LLM models.
Regardless of if you think that buying second hand blood diamonds increases overall demand in the market (which blood diamonds sellers benefit from); it is clearly the case that selling (and reselling) LLM services benefit the LLM providers, and we can trivially see that they’re training new models and not making amends.
I mean yeah: if we went and killed every person who benefits from conflict diamonds and closed all blood diamond mines why wouldn’t you be cool with using the resources? Their evil origin has little to do with their practical utility and if the original sin is expiated there’s no reason not to?
Like yeah conflict diamonds have basically no purpose because we can make diamonds cheaper and better in labs but in a situation where there are more practical uses (cobalt, LLMs) once we cleanse the land of the sinners why wouldn’t we use their ill gotten gains for good?
But we’re not “killing” every person who benefits, literally or figuratively. We’re continuing to buy their diamonds (pay them in money and data) while they continue to mine (train new models, use copyrighted material).
Agreed.
But the entire point I’m making is there’s nothing wrong with the diamonds, the problem is with the method and the people profiting from it.
You were saying the diamonds were not fine by dint of origin. I’m saying let’s right the wrong and then use the diamonds.
It’s not a perfect analogy, models ape the work of artists and take their jobs; it’s like if the diamond was bloody, and as long as it existed, the miner’s family not only didn’t get compensated for the loss but we’re also prevented from getting jobs themselves.
We’re not righting the wrong, were making the wrongs even worse. At some point you have to just burn the whole thing down.
Okay, so again, no new machine learning ever, unless you can prove it’s done without environmental impact or affecting peoples’ right to a dignified existence. That’s the wrong righted. That’s what you’re advocating. Am I misunderstanding?
Do the concepts “reparations” or “compensation for loss and damage” mean anything to you?
The people who were exploited should be the ones to benefit from those diamonds.
The fuck you think “using their ill gotten gains for good” means?
Sounds like you think we should use the diamonds. I wouldn’t be cool with using those diamonds because they belong to the people who were forced to mine them, not me.
I wouldn’t be cool with using those diamonds because they belong to the people who were forced to mine them
well most of them are dead aren’t they? Are we going to put them back into their coffins? Or, how are we planning on redistributing these?
Dead people can’t own things, so that seems like an illogical conclusion, perhaps their estate or family? They didn’t do the work, but they would arguably be most entitled to it.
They had families, so yes, that’d be a good place to start reparations.
The people who were exploited should be the ones to benefit from those diamonds.
i mean probably, but this would be a question of what the law says. If we’re talking philosophy that’s irrelevant here, but to include it anyway, it would be something like “the most ethical source of any given item should be most preferred over any other source of said item” or if we’re operating under an ideology of anti-human exploitation idk where we would even start. You need a really concrete definition of exploitation, and how to combat it effectively, without just exploiting more people.
The law can eat my entire ass.
I see your point and agree with it, but I believe you read more defense in my comment than I tried to put in.
I’m not in favor of this waste.
i mean yeah, statistically, an already mined diamond is a child already dead. you would just stop new diamond mining, or move away from child consumptory diamond mining, you aren’t going to completely demolish every child diamond in existence though, there’s no point, harms already done. Might as well leave them in the market.
But buying them funds the diamond miners to mine more. That’s the point.
By participating in the market, you’re perpetuating blood diamonds.
only if they existed actively. And when it comes to the secondhand market, someone who no longer wants a product, and someone who wants that product, who can buy it from the person who no longer wants it, effectively means that two people get the same value from one singular product, essentially creating value from thin air.
The point i’m making, is that as long as you have diamond mining that requires killing people to produce diamonds, those diamonds are an ethically negative commodity, however if you are no longer killing people, those diamonds are already out there, those people have already been killed, there is nothing you can do to undead those people. So you might as well leave it in market circulation at that point.
You could argue that it would incentivize more illegal mining, but i would argue that bad regulations and enforcement incentivize illegal mining instead.
I think that the market (profit) incentives the mining, and that lax regulations simply fail to curb that incentive. We can see from the war on drugs that simply regulating or criminalizing something won’t stop it if there is a market, it just drives up prices.
But in any case, this aspect is a lot less ambiguous for LLMs because selling the use of an LLM is legal, and the sellers use the money to train new LLMs.
So let’s create regulations around LLMs and enforce those regulations, before it starts to really affect the job market and environment.
i definitely agree that there needs to be some rewriting of law, specifically copyright, which will inevitably be relevant to AI. But i’m still not convinced it’s a massive market black hole or anything.
Unless you want to be cucked like farmers in the US who take in massive subsidies and cry and moan everytime something moderately inconveniences them.
What I want is regulation that protects workers more than employers
Yeah. My phone can generate a 512x512 image with Stable Diffusion in like 20 seconds with the right settings.
Is Midjourney available for use locally now? Or have you misunderstood Midjourney taking 30 seconds to generate from their server as happening locally?
I know a guy that uses Stable Diffusion XL locally with a few LORAs, last time I’ve heard one 2k image took 2 minutes on RTX 30 or 40 something
IDK how expensive is Midjourney in comparison, but running it locally doesn’t sound impossible
The impossible part is that it isn’t publicly available. It’s not an open model.
Oh, ok. I’m not into image generation, so I didn’t know.
That sounds a bit too much. Generating an sdxl image and then scaling it up is the common procedure, but that should not take 2 minutes on a 40xx card. For reference I can generate 3 batches of 5 images (without the upscaling step) in less than 2 minutes on my 4070ti. And that’s without using faster sdxl models like lightning or turbo or whatever.
I swear 50% of Mastodon’s feed is just people having mass hysteria about AI and not understanding how any of it works
Also constantly getting baited about how AI is definitely the reason global warming is happening despite, you know, waves to virtually everything running on fossil fuels
To be fair the cost benefit ratio of llms is way out of whack. It’s even behind outdoor heating in bars.
It’s the same as my enjoyment of video games, 3D modeling, or video editing - my gpu spins up for a few minutes.
Lmao. But ya true, it’s not fair to consider all LLM use equally.
Although I’ve made gpt write nonsense poem for me, most of my prompts are work related that saves me countless hours.
Monkey type inane question, teslacoils power up, screen show smart word words
Ummm, obviously exaggerated but still true, and yeah the fossil fuels are the cause of climate change but AI is driving up the usage.
Honestly it is makes the problem a little worse. It is far from the cause. You can run AI on clean power.
But they aren’t running it on clean power? And so it is a massive contributor?
That’s like saying it’s fine to waste food because it can theoretically be grown sustainably, even when the food you’re wasting wasn’t.
is it a massive contributor? I would venture to argue that air conditioning globally, is probably equally as large, if not larger.
I would argue that most industries, consume more energy. Though it’s also worth noting microsoft spun up the good TMI core to generate power for AI (iirc at the very least a datacenter), so in some cases it is actually consuming cleanly generated power.
That’s like saying it’s fine to waste food because it can theoretically be grown sustainably, even when the food you’re wasting wasn’t.
also to be clear, it’s a little more complicated than this, it doesn’t make sense to ship food products, and then re-ship those food products, only to be shipped again economically, or environmentally, compared to having better local sources of food (and likewise, income, the major problem with global food insecurity is actually monetary, rather than production/consumption based)
Although there are definitely optimizations to be made in terms of how food is consumed. It’s not a black and white, unless you want to live in a strictly controlled environment where you get fed synthetic nutrient paste every few hours. In which case, by all means.
is it a massive contributor? I would venture to argue that air conditioning globally, is probably equally as large, if not larger.
I would argue that most industries, consume more energy. Though it’s also worth noting microsoft spun up the good TMI core to generate power for AI (iirc at the very least a datacenter), so in some cases it is actually consuming cleanly generated power
It’s a new contributor, which means it is contributing on top of already existing contributors like air conditioning and other industries. But even if you include them It’s also particularly bad i.e. data centers powering artificial intelligence could use more electricity than entire cities
Although there are definitely optimizations to be made in terms of how food is consumed. It’s not a black and white, unless you want to live in a strictly controlled environment where you get fed synthetic nutrient paste every few hours. In which case, by all means.
I just want us to live within our energy budget so we don’t cook ourselves.
It’s a new contributor, which means it is contributing on top of already existing contributors like air conditioning and other industries.
yeah but the part that i don’t understand is why this is relevant, sure it’s new, sure it’s not particularly effective but like. Do you really need to be using both the internet and the power grid to shitpost on lemmy right now? You can’t hold arbitrary standards for AI like “it doesnt do anything useful” and then post on one of the least useful websites on the internet with some of the least useful information either.
If we don’t have AI, why don’t we just ban gaming hardware, it just takes up unnecessary production time and consumption costs, as well as electricity. If we’re banning those, why don’t we also just ban most home appliances, those consume a pretty good amount of resources also. And if we’re banning those, why don’t we go a step further and delete housing all together as well, it’s a wasteful industry after all.
Like there’s a point where you start, and a point where you stop, people were doing the same shit over crypto mining, and then the same shit over NFTs, and now nobody gives a fuck about either, tbf sort of dead now, but still. It really just seems like outrage bait to me at best. It starts at “AI bad” because “ai bad” and then stops and “AI bad” because “AI BAD” it’s just illogical reasoning to me.
data centers powering artificial intelligence could use more electricity than entire cities
i really hate this stat, idk why people keep making this statement, you know what uses more electricity than some cities? Some other cities! wow incredible observation, some things use lots of power, and some other things, sometimes also use lots of power.
You mean to tell me a technologically fresh and relevant field has popped onto the market and it’s leading to a massive increase in consumption of electricity, because it consumes electricity? Yeah, that sounds like industry to me.
It’s literally just the mantra of do things and stuff happens. But people are mad about it.
I just want us to live within our energy budget so we don’t cook ourselves.
i’m assuming you’re referring to global warming here, and yeah me too, i would prefer we didn’t end our civilization over some stupid shit like global warming, but i’m more concerned about stuff like dirty energy production, the shipping industry, environmental regulations, things that actually tend to have an effect, rather than like, company consumes lots of energy for a little bit.
BTW, recommend looking into the electrical infrastructure for china, they consume WAY more electricity than the US, and have vastly dirtier production. So if we’re going to complain about AI, go complain about chinese AI training instead.
You mean to tell me a technologically fresh and relevant field has popped onto the market and it’s leading to a massive increase in consumption of electricity, because it consumes electricity? Yeah, that sounds like industry to me.
Yes, and because it’s causing a massive increase in the consumption of electricity it’s causing lots of pollution.
BTW, recommend looking into the electrical infrastructure for china, they consume WAY more electricity than the US, and have vastly dirtier production. So if we’re going to complain about AI, go complain about chinese AI training instead.
🤔
<homer>Alright, but it better be good!</homer>
Simpsonml™️
Don’t worry, we’re not there yet. It can’t even decently generate its own dialog box.
Edit: second version gave me a brain hemmorhage
Continue, or Continue Avatar?
cacol
200x20 ofc
Continue, Avatar.
it really took “box” seriously there
Cacol
That looks a lot better, what did you use? Mine was gpt4 (free account).
Prompt was:
Create a computer dialog box screenshot in mac os aqua style. The title should be “warning”. The message must be “this task will cut off power to a small random city for 10 minutes. Would you like to continue creating this 200x200 avatar?”. The buttons should read “continue” and “cancel”.
Gpt4 uses dall-e under the hood, which is not that great with text.
I’m running Flux locally with ComfyUI.
Yes … power suddenly goes off in their house
The market only cares if the oligarch I am trading with and me are happy with the transaction. Commons is a tragedy for enjoying the lamentations and tears of the commies.
we will push. we will push the button. https://www.youtube.com/watch?v=HoRyrCjOkdI
It’s funny, sad, and kind of true, but to be fair you can create some pretty cool images with AI (think stable-diffusion) even with just mid to high-end consumer hardware.
This is utter fucking bollocks. Fuck all the way off
Do some people just scroll mastodon till they find a meme to crosspost here? Surly federation has a cleaner way to do this?
As far as I know, not even mbin allows crossposting between the microblog side and the thread side with any kind of attribution. But I agree, we should be able to post it to different parts of the ActivityPub world.
The original poster could @ this community, then it would show up here…
Surly federation has a cleaner way to do this?
I don’t know, but even-tenpered federation might. And don’t call me Sir Lee.
Technically possible to do since you have diminishing returns on accuracy by scaling the operation up.
You could use that much power to generate a 200x200 avatar, if a person had that much authority and actually wanted to.
