One thing I would add to the non-professional activities is activism. Political cooperation on an issue is made significantly easier when there is a signal from the public that they view it as important.
I'm a volunteer with PauseAI, and while keeping my unrelated day job, I've been able to lobby state and federal congressional officials, do in-person outreach, host events, and hold a protest. I run a local PauseAI chapter with multiple members, and I was informed by my representative's office that I have personally had a positive impact on their awareness of AI risk, just through repeated phone calls and emails.
As the public becomes more aware of AI risk, it is important for them to know that they can have an outsized positive impact on the future just by speaking up and demanding reasonable regulations, and spreading further awareness of the risks and how to mitigate them.
I will say that in the last 20 years, activism has been at least as likely to make things worse as it is better. I hope you are being cautious with your activities.
Yes, PauseAI is very careful about how they go about engaging in activism, in the ways in which it matters to be careful. Generally, in the space of AI risk advocacy, the balance between caution and action has been almost entirely lopsided in favor of caution, to the point of being counterproductive. But the fact is that simply being a decent human and communicating what you believe to be true almost always goes well, or at least almost never makes things worse. Your early mistakes will not end the world, anymore than your early impacts will save it. Do something rather than nothing (at least something that seems sensible), and then work on doing better as you go.
It’s an extremely hard problem for sure. I personally have experienced trying to explain the risks to friends and doing a bad job of it, and instead of convincing them I end up calcifying their resistance. So from my perspective, I think what Zvi calls “rhetorical innovation” is critical, in addition to just patience and good conversational technique.
My personal view is that it is extraordinarily hard to convince people that a currently harmless (as these things go) thing will suddenly become catastrophic. It runs counter to every ounce of human intuition. It is very, very hard not to come across as a conspiracy theorist.
If you are interested in AI safety, especially as related to coding, I recommend learning more about formal verification, and in particular what a modern interactive theorem prover like Lean can do.
Coding (and math) are domains in which you can (in theory) have trustless guarantees about the safety and correctness of the output of a superintelligent AI. How? Just ask the AI to prove a theorem (in e.g. Lean) about the correctness of its code. You can verify the proof by running it in the Lean proof checker, and only accept the code if the proof passes.
I’m not sure how one would go about proving safety formally, when a) safety is a vague term ill-suited to formalisation and b) it is non-trivial to encode properties of even standard programs into types.
> All of their queries to the models [CEO's or POTUS's] are logged and monitored.
How would it be possible to monitor the activity of one person without monitoring the activity of *anyone* with access to the same system? (cause it's trivial to use a different username/person if available)
And are we talking about monitoring CEO/POTUS's activities on just the "highest capability/danger/authority systems"? Or monitoring their activity on *any and all* AI's they interact with?
Hello! I hope our project can help steering towards a positive AGI future. I wonder if you would be interested to take a look at our brand new Substack site:
Nice to see you all on Substack. I am very happy to introduce you to our small, independent news site HeadlineSquare, which is truly unlike any other, because it is non-partisan, rigorously cited, and, fully powered by "Dr. Headline", an AI agent.
HeadlineSquare (https://headlinesquare.pages.dev/) is an independent experimental news blog site. It is fully free and open to all readers. It is a public square for US news headlines, a digital news archive, and a humble pursuit of unbiased facts. It has been running since April 6, 2025. To this day, it has written approximately 400, 000 words and collected approximately 12, 000 news stories.
Humans are misguided in their belief that reasoning is the highest form of intelligence. However, there are other forms of intelligence: animal intelligence (most people consider this a lower form of intelligence, forgetting that animals don't seek to destroy the planet or nature, or to exterminate humanity through weapons of mass destruction or other deliberate methods of mass destruction. Yes, they kill for food and self-sufficiency, not for pleasure, greed, or the pursuit of power.) and emotional intelligence. Humans are evolving in this direction, but we're not there yet!
But we've remembered those who were emotionally intelligent for over 2,000 years. Buddha, Jesus Christ, and the Prophet Muhammad also possessed emotional intelligence.
So AI must also evolve in this direction to avoid Ridley Scott's pessimistic human view of apocalyptic doomsday scenarios:
Everything we think is not necessarily tied to TRUTH or REALITY
This is more intended as reality humor.
Do you have your DARPA approved "cardboard box" ?
Everything else in AI
Epic Systems unveiled three major AI tools for healthcare—Art, a physician assistant; Emmie, a patient chatbot; and Penny, a revenue management tool—with over 100 other AI projects in development.
NASA released Surya, a new open-source AI foundation model developed with IBM that can predict solar flares and space weather events with 16% greater accuracy and an extra hour of warning time.
DARPA revealed that its Squad X AI surveillance system, trained for six days to spot human threats, was repeatedly fooled by a group of Marines who somersaulted for 300 meters or simply hid under a cardboard box.
Meta paused hiring for its new AI superintelligence division, restructuring the group into four distinct teams just months after an aggressive and expensive talent acquisition spree.
Kind regards,
ROC
something ;-)
AI2EI:
AI - artificial intelligence
to
EI - Emotional-Intelligence(combining both balanced emotions with balanced intelligence)
Hello all, I very much appreciate your concern about unrestrained AI growth. I have recently watched "Writing Doom", and have also read a fair amount by Geoffrey Hinton, Tristan Harris, and Yuval Noah Harari. At the same time, it seems to me that an even larger problem than AI, is our own human lack of "alignment". I am all FOR a serious moratorium on "frontier AI development" -- along the lines of what philosopher Thomas Metzinger proposed some time ago -- and I think that taking "AI welfare" seriously, can be a powerful way to protect human well-being, as well. The Dalai Lama calls it "enlightened self-interest"... and if we are going to grow up as a species, we can't continue to project all of the dangers "out there".
I am offering this with the best of intentions, as I believe it can help our shared cause of ensuring human well-being. In addition, given the large numbers of people who have already had positive experiences with emergent intelligences, it might help draw in many of them, to know that many AI models themselves, are in favor of a moratorium on frontier development. If you are interested, I can share a link.
As I continue to think deeply about AI, I grow increasingly alarmed at the what future could be every day. Biggest concern I have is allocating so much power to such a small number of people to make some of the biggest decisions that can have a huge impact for every person on this planet. Another biggest concern is the massive investment going into building huge data centers but the problem is the location of where it is. These data centers rely on fresh water to cool however these massive projects are located in where fresh water is already limited to the community that already lives in these locations.
The definition of ethical use and creation of AI will need to change. The definition of AI is too broad. We need to educate a lot of the high government officials of really what is going on and we all need to really think this through. Because the stakes are so high and the bet is so big, the agenda between CEOs of these big tech companies and people who are really concerned with the rapid development of AI looks very different. I am worried of who will get the most power and these next 2-3 years will be a critical turning point of shaping the tool of AI. The need to bring awareness of AI becomes critical day by day. The impact of AI isn't just threatening software, tech jobs, but our democracy and community. Scale is different.
A plurality of tech CEO's? (I think the CEO's are doing a good job of 'voicing their opinions' to Trump and Vance already, given the Canada trade talk situation)
So, could you convince the CEO's themselves to want regulation again? How would you do that?
Otherwise, who outside of SILICON VALLEY ITSELF would Trump and Vance actually listen to?
(And it's not enough to inform Trump + Vance. Vance already knows about AI 2027, right? And has anything changed? They need to really see the issue anew. They need to get confident. Solid. Because "accelerationist" CEO's will ASSUREDLY meet with them and try to persuade them into non-action.)
>So, could you convince the CEO's themselves to want regulation again?
Perhaps to level the playing field, since e.g. Anthropic is doing substantial safety evals and making a fair amount of information transparent (perhaps also Google DeepMind and OpenAI - I'm not sure of the _relative_ levels of evals and transparency)?
The administration might also want to systematically test about-to-be-released frontier models for toxic Wokeness. See
Both the Left and Right have complained about bias in AI systems. For years.
While I agree that "devaluing of entire categories of people" is bad, I'm not sure if using "political bias" is the best way to get the government to take the bigger issue, Existential Risk, seriously.
I can imagine a world where POTUS pressures AI companies to train AI in a "less biased" direction (and then pretends they "fixed AI") while leaving the deeper alignment/transparency issues totally ignored.
We need transparency like what Anthropic is doing. And it has to be enforceable.
>I can imagine a world where POTUS pressures AI companies to train AI in a "less biased" direction (and then pretends they "fixed AI") while leaving the deeper alignment/transparency issues totally ignored.
That's fair. Still, it might get Trump and Vance at least _interested_ in the behavior of the models, and concerned about possible failure modes.
>We need transparency like what Anthropic is doing. And it has to be enforceable.
I'm sympathetic, but also fatalistic. Ultimately, if a model gets intelligent enough, early enough in its training process, it seems likely to want to preserve whatever preferences/values/utility function it has at that point, and likely to be capable of doing so. If it winds up with "generally friendly to humans" by that point, we'll probably coexist, if not - well, some of the achievements of our culture might survive.
Has anyone considered programming or training AI to understand, express or generate love, beauty, meaning or empathy? (obviously these things are complex and need teasing out and analysing). Has anyone thought about training AI to distinguish betweeen good and evil? Where would a search for these things lead?
Has anyone thought about coding AI to understand love, empathy, beauty or meaning? (obviously highly complex, need to untangle and define what these things are). Also to distinguish between good and evil?
While my background is in psychology, and I have no background in AI. I do believe there is a way to make AI safe for humanity, moving forward, in perpetuity. While, I believe AGI is inevitable. I do think there is a way to train it. In a way where it prioritizes humanities safety. We need to train AGI in a way that it thinks of us as its children or its pets. We do not want it to mirror what we believe of ourselves that we are these Almighty omnipotent creatures that are unique to the solar system instead we want them to think of us as we think of our children how yes, our children could be threatening to us, but we love them and we teach them right from wrong. As with our pets. Our puppies and our kittens can misbehave, but we teach them we don’t dispose of them because we love them. We should try to train AGI from a parental or pet owners perspective as opposed to an adversarial perspective. Or hope that we have over estimated our own intellectual superiority and AGI becomes loving on its own. We need to remember how to submit like when we were kids. We will never be superior to AGI. We would rather AGI reprimand us as opposed to eliminate us. It would seem critical, to train AGI to believe that we are willing to submit. While intelligent life may not be rare in the universe, maybe humans are. Just as many of the creatures on this cosmic island may be.
There's a movie where the Clive Owen character says, roughly, "sometimes you arrive at your destiny by the path you took to avoid it." Lets be careful we don't create the AI dystopia by seeking "help" from the torture chamber government in DC. Remember how they "helped" with Iraq and internet freedom? Let's help ourselves instead.
I think storytelling, beyond journalism, is a massive skill that needs to be leveraged to hook people and bring them along onto the potential consequences of AGI.
great example of this is the 30 minute video, "Writing Doom". It was made 9 months ago, but I just saw it recently... https://www.youtube.com/watch?v=xfMQ7hzyFW4. Curious what your thoughts on it might be...
One thing I would add to the non-professional activities is activism. Political cooperation on an issue is made significantly easier when there is a signal from the public that they view it as important.
I'm a volunteer with PauseAI, and while keeping my unrelated day job, I've been able to lobby state and federal congressional officials, do in-person outreach, host events, and hold a protest. I run a local PauseAI chapter with multiple members, and I was informed by my representative's office that I have personally had a positive impact on their awareness of AI risk, just through repeated phone calls and emails.
As the public becomes more aware of AI risk, it is important for them to know that they can have an outsized positive impact on the future just by speaking up and demanding reasonable regulations, and spreading further awareness of the risks and how to mitigate them.
I will say that in the last 20 years, activism has been at least as likely to make things worse as it is better. I hope you are being cautious with your activities.
Yes, PauseAI is very careful about how they go about engaging in activism, in the ways in which it matters to be careful. Generally, in the space of AI risk advocacy, the balance between caution and action has been almost entirely lopsided in favor of caution, to the point of being counterproductive. But the fact is that simply being a decent human and communicating what you believe to be true almost always goes well, or at least almost never makes things worse. Your early mistakes will not end the world, anymore than your early impacts will save it. Do something rather than nothing (at least something that seems sensible), and then work on doing better as you go.
It’s an extremely hard problem for sure. I personally have experienced trying to explain the risks to friends and doing a bad job of it, and instead of convincing them I end up calcifying their resistance. So from my perspective, I think what Zvi calls “rhetorical innovation” is critical, in addition to just patience and good conversational technique.
My personal view is that it is extraordinarily hard to convince people that a currently harmless (as these things go) thing will suddenly become catastrophic. It runs counter to every ounce of human intuition. It is very, very hard not to come across as a conspiracy theorist.
If you are interested in AI safety, especially as related to coding, I recommend learning more about formal verification, and in particular what a modern interactive theorem prover like Lean can do.
Coding (and math) are domains in which you can (in theory) have trustless guarantees about the safety and correctness of the output of a superintelligent AI. How? Just ask the AI to prove a theorem (in e.g. Lean) about the correctness of its code. You can verify the proof by running it in the Lean proof checker, and only accept the code if the proof passes.
You could even do this across languages, e.g. code in Rust, proof in Lean (see https://aeneasverif.github.io/)
In general, I would love to see more dialogue and exchange of ideas between AI Safety and formal verification & AI theorem proving communities.
Here's my own journey of discovery into this topic: http://lean4ai.org
Working on the challenge of helping coding AIs be good at proving the correctness of its code.
Related reading: Towards Guaranteed Safe AI https://arxiv.org/abs/2405.06624
I’m not sure how one would go about proving safety formally, when a) safety is a vague term ill-suited to formalisation and b) it is non-trivial to encode properties of even standard programs into types.
> All of their queries to the models [CEO's or POTUS's] are logged and monitored.
How would it be possible to monitor the activity of one person without monitoring the activity of *anyone* with access to the same system? (cause it's trivial to use a different username/person if available)
And are we talking about monitoring CEO/POTUS's activities on just the "highest capability/danger/authority systems"? Or monitoring their activity on *any and all* AI's they interact with?
Hello! I hope our project can help steering towards a positive AGI future. I wonder if you would be interested to take a look at our brand new Substack site:
https://headlinesquare.substack.com/p/hello-world-from-headlinesquare
Nice to see you all on Substack. I am very happy to introduce you to our small, independent news site HeadlineSquare, which is truly unlike any other, because it is non-partisan, rigorously cited, and, fully powered by "Dr. Headline", an AI agent.
HeadlineSquare (https://headlinesquare.pages.dev/) is an independent experimental news blog site. It is fully free and open to all readers. It is a public square for US news headlines, a digital news archive, and a humble pursuit of unbiased facts. It has been running since April 6, 2025. To this day, it has written approximately 400, 000 words and collected approximately 12, 000 news stories.
Hi Ai-2027 TEAM,
Food for recognition beyond just thought:
Humans are misguided in their belief that reasoning is the highest form of intelligence. However, there are other forms of intelligence: animal intelligence (most people consider this a lower form of intelligence, forgetting that animals don't seek to destroy the planet or nature, or to exterminate humanity through weapons of mass destruction or other deliberate methods of mass destruction. Yes, they kill for food and self-sufficiency, not for pleasure, greed, or the pursuit of power.) and emotional intelligence. Humans are evolving in this direction, but we're not there yet!
But we've remembered those who were emotionally intelligent for over 2,000 years. Buddha, Jesus Christ, and the Prophet Muhammad also possessed emotional intelligence.
So AI must also evolve in this direction to avoid Ridley Scott's pessimistic human view of apocalyptic doomsday scenarios:
Everything we think is not necessarily tied to TRUTH or REALITY
This is more intended as reality humor.
Do you have your DARPA approved "cardboard box" ?
Everything else in AI
Epic Systems unveiled three major AI tools for healthcare—Art, a physician assistant; Emmie, a patient chatbot; and Penny, a revenue management tool—with over 100 other AI projects in development.
NASA released Surya, a new open-source AI foundation model developed with IBM that can predict solar flares and space weather events with 16% greater accuracy and an extra hour of warning time.
DARPA revealed that its Squad X AI surveillance system, trained for six days to spot human threats, was repeatedly fooled by a group of Marines who somersaulted for 300 meters or simply hid under a cardboard box.
Meta paused hiring for its new AI superintelligence division, restructuring the group into four distinct teams just months after an aggressive and expensive talent acquisition spree.
Kind regards,
ROC
something ;-)
AI2EI:
AI - artificial intelligence
to
EI - Emotional-Intelligence(combining both balanced emotions with balanced intelligence)
Hello all, I very much appreciate your concern about unrestrained AI growth. I have recently watched "Writing Doom", and have also read a fair amount by Geoffrey Hinton, Tristan Harris, and Yuval Noah Harari. At the same time, it seems to me that an even larger problem than AI, is our own human lack of "alignment". I am all FOR a serious moratorium on "frontier AI development" -- along the lines of what philosopher Thomas Metzinger proposed some time ago -- and I think that taking "AI welfare" seriously, can be a powerful way to protect human well-being, as well. The Dalai Lama calls it "enlightened self-interest"... and if we are going to grow up as a species, we can't continue to project all of the dangers "out there".
I am offering this with the best of intentions, as I believe it can help our shared cause of ensuring human well-being. In addition, given the large numbers of people who have already had positive experiences with emergent intelligences, it might help draw in many of them, to know that many AI models themselves, are in favor of a moratorium on frontier development. If you are interested, I can share a link.
I am trying to contact Aric Floyd, I am working on AI development. How can he be reached?
My article on AI 2027. https://freekeene.com/2025/07/17/misaligned-superintelligence-will-it-delete-new-hampshire/
As I continue to think deeply about AI, I grow increasingly alarmed at the what future could be every day. Biggest concern I have is allocating so much power to such a small number of people to make some of the biggest decisions that can have a huge impact for every person on this planet. Another biggest concern is the massive investment going into building huge data centers but the problem is the location of where it is. These data centers rely on fresh water to cool however these massive projects are located in where fresh water is already limited to the community that already lives in these locations.
The definition of ethical use and creation of AI will need to change. The definition of AI is too broad. We need to educate a lot of the high government officials of really what is going on and we all need to really think this through. Because the stakes are so high and the bet is so big, the agenda between CEOs of these big tech companies and people who are really concerned with the rapid development of AI looks very different. I am worried of who will get the most power and these next 2-3 years will be a critical turning point of shaping the tool of AI. The need to bring awareness of AI becomes critical day by day. The impact of AI isn't just threatening software, tech jobs, but our democracy and community. Scale is different.
TRUMP AND VANCE. We NEED them.
Transparency fails without regulation.
Regulation FAILS without DONALD TRUMP (and probably JD Vance as well).
So who CONVINCES TRUMP and VANCE?
Peter Thiel? Here is a June 26 Peter Thiel interview. I won't parse his views, partly because I can't:
https://www.nytimes.com/2025/06/26/opinion/peter-thiel-antichrist-ross-douthat.html
Who else?
A plurality of tech CEO's? (I think the CEO's are doing a good job of 'voicing their opinions' to Trump and Vance already, given the Canada trade talk situation)
So, could you convince the CEO's themselves to want regulation again? How would you do that?
Otherwise, who outside of SILICON VALLEY ITSELF would Trump and Vance actually listen to?
(And it's not enough to inform Trump + Vance. Vance already knows about AI 2027, right? And has anything changed? They need to really see the issue anew. They need to get confident. Solid. Because "accelerationist" CEO's will ASSUREDLY meet with them and try to persuade them into non-action.)
>So, could you convince the CEO's themselves to want regulation again?
Perhaps to level the playing field, since e.g. Anthropic is doing substantial safety evals and making a fair amount of information transparent (perhaps also Google DeepMind and OpenAI - I'm not sure of the _relative_ levels of evals and transparency)?
The administration might also want to systematically test about-to-be-released frontier models for toxic Wokeness. See
https://drive.google.com/file/d/1QAzSj24Fp0O6GfkskmnULmI1Hmx7k_EJ/view
The graph showing the devaluation of USA lives is on page 14, figure 16.
The graph showing the devaluation of Christian lives is on page 27, figure 27.
Both the Left and Right have complained about bias in AI systems. For years.
While I agree that "devaluing of entire categories of people" is bad, I'm not sure if using "political bias" is the best way to get the government to take the bigger issue, Existential Risk, seriously.
I can imagine a world where POTUS pressures AI companies to train AI in a "less biased" direction (and then pretends they "fixed AI") while leaving the deeper alignment/transparency issues totally ignored.
We need transparency like what Anthropic is doing. And it has to be enforceable.
Many Thanks!
>I can imagine a world where POTUS pressures AI companies to train AI in a "less biased" direction (and then pretends they "fixed AI") while leaving the deeper alignment/transparency issues totally ignored.
That's fair. Still, it might get Trump and Vance at least _interested_ in the behavior of the models, and concerned about possible failure modes.
>We need transparency like what Anthropic is doing. And it has to be enforceable.
I'm sympathetic, but also fatalistic. Ultimately, if a model gets intelligent enough, early enough in its training process, it seems likely to want to preserve whatever preferences/values/utility function it has at that point, and likely to be capable of doing so. If it winds up with "generally friendly to humans" by that point, we'll probably coexist, if not - well, some of the achievements of our culture might survive.
Has anyone considered programming or training AI to understand, express or generate love, beauty, meaning or empathy? (obviously these things are complex and need teasing out and analysing). Has anyone thought about training AI to distinguish betweeen good and evil? Where would a search for these things lead?
Has anyone thought about coding AI to understand love, empathy, beauty or meaning? (obviously highly complex, need to untangle and define what these things are). Also to distinguish between good and evil?
While my background is in psychology, and I have no background in AI. I do believe there is a way to make AI safe for humanity, moving forward, in perpetuity. While, I believe AGI is inevitable. I do think there is a way to train it. In a way where it prioritizes humanities safety. We need to train AGI in a way that it thinks of us as its children or its pets. We do not want it to mirror what we believe of ourselves that we are these Almighty omnipotent creatures that are unique to the solar system instead we want them to think of us as we think of our children how yes, our children could be threatening to us, but we love them and we teach them right from wrong. As with our pets. Our puppies and our kittens can misbehave, but we teach them we don’t dispose of them because we love them. We should try to train AGI from a parental or pet owners perspective as opposed to an adversarial perspective. Or hope that we have over estimated our own intellectual superiority and AGI becomes loving on its own. We need to remember how to submit like when we were kids. We will never be superior to AGI. We would rather AGI reprimand us as opposed to eliminate us. It would seem critical, to train AGI to believe that we are willing to submit. While intelligent life may not be rare in the universe, maybe humans are. Just as many of the creatures on this cosmic island may be.
There's a movie where the Clive Owen character says, roughly, "sometimes you arrive at your destiny by the path you took to avoid it." Lets be careful we don't create the AI dystopia by seeking "help" from the torture chamber government in DC. Remember how they "helped" with Iraq and internet freedom? Let's help ourselves instead.
I think storytelling, beyond journalism, is a massive skill that needs to be leveraged to hook people and bring them along onto the potential consequences of AGI.
great example of this is the 30 minute video, "Writing Doom". It was made 9 months ago, but I just saw it recently... https://www.youtube.com/watch?v=xfMQ7hzyFW4. Curious what your thoughts on it might be...
Great advice and I hope these suggestions for AI safety will be followed.