Earlier this week, a former OpenAI employee accused the company of being careless about AI development. On Thursday, OpenAI released a research paper that shows how the workings of an AI model, specifically GPT-4, can be reverse engineered. Basically, the company wants to make AI models more easily explainable, so that people know that the company is serious about tackling AI risks. Of course, OpenAI says that the new tool is aimed at making the advanced functionalities of GPT-4 more accessible and understandable to researchers, developers, and enthusiasts in the field of AI.

The primary purpose of this new tool is to understand the inner workings of GPT-4 by breaking down its complex model into understandable segments. AI tools like GPT-4 are sophisticated and complex models due to its ability to generate human-like responses. This ability often leaves users confused about how it arrives at specific outputs. OpenAI’s new tool is basically aimed at extracting meaningful concepts from the huge and complex structure of GPT-4, making it easier to understand and use.

The research paper details sparse autoencoders and explores techniques for creating efficient representations of data. Sparse autoencoders are a type of neural network that aims to learn compressed, useful features from input data while promoting sparsity in their activation patterns. This means that only a small number of neurons are activated at a time, which enhances the model’s ability to identify important structures in the data. The paper discusses various ways to implement sparsity and demonstrates how these techniques can improve performance in tasks such as image recognition and data reconstruction.

OpenAI claims the tool will enhance the usability of GPT-4. It says developers and researchers can now explore the model’s output more effectively, allowing them to fine-tune applications and create more accurate and reliable AI systems.

While the research paper outlines OpenAI’s commitment to AI safety, it also draws attention to the company’s recent internal issues. The new study was conducted by OpenAI’s now-disbanded “Superalignment” team, which focused on the long-term risks of AI. The paper’s co-authors include former co-leaders Ilya Sutskever and Jan Leakey, who have both now left OpenAI. Interestingly, co-founder and former chief scientist Sutskever was involved in the controversial decision to oust CEO Sam Altman sometime last November.