* You have to pinch yourself sometimes but we really are approaching Terminator land. I'm hoping we can keep AI away from those kind of destructive systems.
* I find the constitution AI method super interesting because I never would connect technology to society building, but here we are. I wonder if that training has to be ongoing and interestingly, subject to a democratic procedure. Just like we elect judges, congress people, etc who ultimately draft, change, and interpret law - do we something similar here? Fascinating to think about.
* As someone who only knows AI from the practitioner side, I find the Explainability part most haunting. I'm an engineer - I've always been able to reverse engineer every output. This is the first time where I truly don't know how output is being formed. It's so strange. And what's wild is that the creators of these LLMs feel the same way. Stephen Wolfram gets at this in his "What is ChatGPT Doing..." piece here: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
Agreed all around! It's cool that we now have more direct mechanisms to take societal values and infuse them in our technology. It'll be so interesting when there's a well-encapsulated profit-maximizing motive and flourishing-maximizing motive, and then we have to debate how much we turn the dial between the two (50/50? 90/10? Profit only when it doesn't conflict with flourishing?), and how that might manifest in terms of real policies
It's definitely wild we don't actually know what's happening in there. Chris Olah said on a podcast how wild it is that it's easier for us to make something that automatically figures out how to do something than it is to actually do the thing ourselves. Instead of baking a cake you can make a system that auto-learns how to bake a cake better than we could! And it doesn't even tell us how!
If this bet around "LLM by vote" pans out, I wonder if there's an interesting opportunity to create that layer between people and the LLMs. Rather than weights that go up and down by the LLM creators, it's society input => training => adjusted models.
My takeaways from your article:
* You have to pinch yourself sometimes but we really are approaching Terminator land. I'm hoping we can keep AI away from those kind of destructive systems.
* I find the constitution AI method super interesting because I never would connect technology to society building, but here we are. I wonder if that training has to be ongoing and interestingly, subject to a democratic procedure. Just like we elect judges, congress people, etc who ultimately draft, change, and interpret law - do we something similar here? Fascinating to think about.
* As someone who only knows AI from the practitioner side, I find the Explainability part most haunting. I'm an engineer - I've always been able to reverse engineer every output. This is the first time where I truly don't know how output is being formed. It's so strange. And what's wild is that the creators of these LLMs feel the same way. Stephen Wolfram gets at this in his "What is ChatGPT Doing..." piece here: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
Agreed all around! It's cool that we now have more direct mechanisms to take societal values and infuse them in our technology. It'll be so interesting when there's a well-encapsulated profit-maximizing motive and flourishing-maximizing motive, and then we have to debate how much we turn the dial between the two (50/50? 90/10? Profit only when it doesn't conflict with flourishing?), and how that might manifest in terms of real policies
It's definitely wild we don't actually know what's happening in there. Chris Olah said on a podcast how wild it is that it's easier for us to make something that automatically figures out how to do something than it is to actually do the thing ourselves. Instead of baking a cake you can make a system that auto-learns how to bake a cake better than we could! And it doesn't even tell us how!
If this bet around "LLM by vote" pans out, I wonder if there's an interesting opportunity to create that layer between people and the LLMs. Rather than weights that go up and down by the LLM creators, it's society input => training => adjusted models.