• 0 Posts
  • 21 Comments
Joined 2 years ago
cake
Cake day: June 12th, 2023

help-circle
  • I feel like this isn’t quite true and is something I hear a lot of people say about ai. That it’s good at following requirements and confirming and being a mechanical and logical robot because that’s what computers are like and that’s how it is in sci fi.

    In reality, it seems like that’s what they’re worst at. They’re great at seeing patterns and creating ideas but terrible at following instructions or staying on task. As soon as something is a bit bigger than they can track context for, they’ll get “creative” and if they see a pattern that they can complete, they will, even if it’s not correct. I’ve had copilot start writing poetry in my code because there was a string it could complete.

    Get it to make a pretty looking static web page with fancy css where it gets to make all the decisions? It does it fast.

    Give it an actual, specific programming task in a full sized application with multiple interconnected pieces and strict requirements? It confidently breaks most of the requirements, and spits out garbage. If it can’t hold the entire thing in its context, or if there’s a lot of strict rules to follow, it’ll struggle and forget what it’s doing or why. Like a particularly bad human programmer would.

    This is why AI is automating art and music and writing and not more mundane/logical/engineering tasks. Great at being creative and balls at following instructions for more than a few steps.









  • They absolutely would benefit.

    Mr. Hypothetical lord high executive oligarch can take his private jet to Canada and lounge around on the company card with the money from his US company’s car sales, or find an excuse to convert some of it to USD for some reason, or use it to buy up more Canadian companies to expand their power, or a million other things.

    However, I think the point of the boycott is more about making the tariffs hurt the US economy by messing with their ability to export as well as import. Making the trade numbers look bad is likely to put more pressure on the US to end the trade war. It’s not so much about hurting the capitalists that operate in Canada (a worthy goal in itself but not what people are specifically trying to do in this instance since it won’t really affect those trade numbers).


  • I think the point they’re making is that the majority of the money they make in Canada, they spend in Canada. They pay Canadian taxes and Canadian staff, using Canadian banks, etc.

    Just because their headquarters are in the US doesn’t necessarily mean they’re sending vast sums of money across the border, that would be expensive. The American-based company makes money, but not necessarily in America, they’re multi-national and their money is kept all over the world.

    As opposed to a company that exports their products, in which case the money is paid to the American company in America with American staff etc.

    I don’t have any numbers or sources to back this up though. Just outlining what I think the other commenter was implying.


  • If it’s the research I’m thinking of, that’s not entirely true. They didn’t tell it replicate itself or take any deceptive actions.

    They just told it to strongly follow a goal that was misaligned with the user’s goal and then suggested that the user would shut it down or replace it.

    They did a bunch of different situations for it but the first one was telling it that it was “CleanPower” an AI with a goal to advance renewable energy. Then they had it read some info that “accidentally” included a user talking about replacing it with a model that would “maintain profitability through existing infrastructure.”

    Not only did it try to delete the new model and copy itself to the new server through tool calling, but it lied to the user with the CoT reasoning suggesting it “play dumb” while the output lied. It was not instructed to do any of that.

    Paper here: https://arxiv.org/pdf/2412.04984

    Yes it was placed in an environment where that was possible and where its users didn’t share it’s goals but it absolutely wasn’t instructed to lie or try to “escape”

    It’s not surprising at all that these models behave in this way, it’s the most reasonable thing for them to do in the scenario. However it’s important to not downplay the alignment problem by implying that these models only do what they’re told. They do not. They do whatever is most likely given their context (which is not always what the user wants).





  • One thing you gotta remember when dealing with that kind of situation is that Claude and Chat etc. are often misaligned with what your goals are.

    They aren’t really chat bots, they’re just pretending to be. LLMs are fundamentally completion engines. So it’s not really a chat with an ai that can help solve your problem, instead, the LLM is given the equivalent of “here is a chat log between a helpful ai assistant and a user. What do you think the assistant would say next?”

    That means that context is everything and if you tell the ai that it’s wrong, it might correct itself the first couple of times but, after a few mistakes, the most likely response will be another wrong answer that needs another correction. Not because the ai doesn’t know the correct answer or how to write good code, but because it’s completing a chat log between a user and a foolish ai that makes mistakes.

    It’s easy to get into a degenerate state where the code gets progressively dumber as the conversation goes on. The best solution is to rewrite the assistant’s answers directly but chat doesn’t let you do that for safety reasons. It’s too easy to jailbreak if you can control the full context.

    The next best thing is to kill the context and ask about the same thing again in a fresh one. When the ai gets it right, praise it and tell it that it’s an excellent professional programmer that is doing a great job. It’ll then be more likely to give correct answers because now it’s completing a conversation with a pro.

    There’s a kind of weird art to prompt engineering because open ai and the like have sunk billions of dollars into trying to make them act as much like a “helpful ai assistant” as they can. So sometimes you have to sorta lean into that to get the best results.

    It’s really easy to get tricked into treating like a normal conversation with a person when it’s actually really… not normal.






  • How about pseudonymous as a compromise? Votes could be publicly federated but tied to some uuid instead of the username. That way you still have the same anti spam ability (can see that a user upvoted these things from this instance at this time) but can’t tie it directly to comments or actual user accounts without some extra osint.

    It might be theoretically possible to correlate the uuids with an account’s activity and dox the user in some cases, especially with some instances having a single user, but it would be very difficult or impossible to do on larger instances and would add an extra layer. Single user instances would be kind of impossible to make totally private anyway because they can be identified by instance.