On September 29 and following a solicitation for community inputs, the AGMAI issued recommendations to AI companies regarding the release of their mathematical results. The underlying dilemma is not a novel one: what should the scientific community do with results that were obtained unethically? By offering guidelines, the AGMAI sought to mitigate the harm of such releases; but it also risked offering AI companies a roadmap for the continued strip-mining of mathematics, now with the fig leaf of adhering to community-approved “best practices” for processing the resulting ores.
Then, on October 6, OpenAI released a repository of files purporting to contain solutions to a number of high-profile problems in mathematics. In the spirit of continued debate and discussion within the mathematical community on how to address such releases, we are sharing with the public a selection of the community comments made by our members to AGMAI in advance of their public statement (out of more than thirty such comments shared with us by members). Some were lightly edited for brevity. These comments showcase a number of perspectives which were not represented in AGMAI’s recommendations.
Alessandro Della Corte, Università degli Studi di Camerino:
In my opinion, a “good way” no longer exists: when too many wrong choices are made from the standpoint of scientific ethics (and I would say ethics tout court), a “good” way out may no longer exist. Moreover, I don't believe that trying to “help” an entity that was openly hostile up to yesterday, that is infinitely more powerful economically than any mathematical institution and that enjoys political support at the highest level makes the slightest sense. The mathematical community’s real power is not money or politics, it's epistemic authority, and I think whatever the Advisory Board will do, OpenAI will end up taking from it the exact thing it has taken from the math community: our soft-power.
Alexis Marchand, IM PAM:
Dear AGMAI,
I am very concerned with the acceleration of the technical development of large language models in the hands of a few private companies, not only for its impact on mathematics, but also on society at large. I would like to use our voice as mathematicians to demand from OpenAI and other LLM companies that they put a halt, or at least slow down, the technical development of LLMs, and contribute to financing public research (which must be carried out independently of those companies) on (1) AI safety, (2) The societal impact of AI and how to mitigate it (in mathematics and elsewhere) and (3) The climatic impact of AI, and how to choose which uses should have priority.
Lorenzo Riva, Harvard University CMSA:
Dear AGMAI,
As you discuss possible suggestions for OpenAI to release and disseminate their results, I urge you to consider the following question and its many derivatives: What will happen next?
1. I am sure that you’ve considered the fate of graduate students. These 100+ open problems may constitute partial or full theses in progress for thousands of graduate students across the world. What should they do after OpenAI posts their solutions? I shouldn’t even point this out, but note that this situation differs by several orders of magnitude with that of a normal mathematician accidentally scooping someone’s thesis. For starters, mathematicians tend to share the problems they are working on, and also take longer to write them up, and might even consider delaying such publications if it is in the best interest of a younger colleague; I’m not sure OpenAI would do us such a favor.
2. Let’s say that senior faculty (or worse, university admins), smitten by the wealth of solved problems, decide that there is no point in hiring graduate students to advance mathematical knowledge since AI users could do all the work, check it, and publish it in a fraction of the time. How long will mathematics be able to remain a respectable discipline? How many mathematicians will there be in 50 years? What will we do when the next model autonomously decides that math doesn’t need humans anymore? Maybe you’re of the opinion that mathematics, the discipline, will thrive; but even you must admit that mathematicians, the humans, will not. Do you have any guarantees from OpenAI that this won’t happen?
3. Why should OpenAI stop at 100 problems, after you provide them with an opportunity to share them? Why not 1000, or 5000, or whatever number will appease their future shareholders? Will an advisory group need to be formed every time a corporation decides to use mathematics as a punching bag for their technologies? At what point can we stop trying to be politicians and go back to working on our research problems (those few remaining open, at least)?
I am confident that you have the best intentions in mind and that you are very concerned about the future of mathematics. Therefore the best recommendation you can offer an AI company is to not release any results and to cease all current work on open problems in mathematics, at least for the foreseeable future. You have some limited power to influence these corporations and it is your duty, both as well-respected senior members of the community and as ambassadors, to use this power for the benefit of mathematicians around the world.
Evan Scott, CUNY Graduate Center:
I write as one of over 600 members of the Association for Human Mathematics. Mathematics has long run on a social contract among its researchers. One is expected to communicate with others in one's field (to avoid accidentally scooping people), to present one's work with humility and respect to others, to offer to involve other researchers in a project if one learns key ideas from them (to avoid stealing those ideas), to write readable work that can easily be checked for correctness and understood by others, and a million other rules, courtesies, and so on. The "punishment" for breaking these rules is lost social standing, which severely impacts one's ability to do and to publicize good work. Indeed, our field does not operate as a meritocracy. It operates as a society with meritocratic ideals.
Large AI companies plainly do not see themselves as signatories of this social contract. OpenAI and other large companies have flagrantly disrespected these rules – scooping people, presenting their work in grandiose terms, writing papers whose correctness is impossible to check and whose ideas are made opaque by mountains of AI-generated text, and (with respect to Navier-Stokes) perhaps outright stealing from other researchers already hard at work.
It is critical that AGMAI impress upon these companies their bonds to the social contract, and if they refuse (as many expect them to) it is critical that AGMAI publicly withdraws its support of OpenAI. Any breach of the social contract which is “permitted” by anyone damages the future of that contract for everyone, destroying social trust and exposing us all to increased bad behaviour from other mathematicians.
I have spent the last five years of blood, sweat, tears, and great effort trying to become a member of the mathematics community and to get a job in this field. I will graduate with my PhD next year or the year after, and I am deeply afraid that between now and then the social contract I have paid into will be destroyed by established mathematicians acting with insufficient foresight. You have a grave responsibility. Please do not betray the effort and sacrifices that I and many others have poured into this magical and unique field.
Tasmin Chu, Caltech
It is highly disturbing that OpenAI is once again foraying into research mathematics, particularly when their conduct raises serious safety concerns. Indeed, OpenAI disclosed that over 10 000 agents were deployed to purportedly resolve the Navier-Stokes problem; meanwhile, only 1200 AI agents of an earlier model were involved in the HuggingFace incident, which resulted in federal cybersecurity crimes. These crimes do not appear to be isolated incidents. In June, OpenAI agents hacked an Australian health portal and accessed private statistical data; OpenAI did not discover this incident until August, and did not inform the Australian government until September.
It is worth asking why those who release AI-generated results actually believe them. For instance, the Navier-Stokes document was purportedly produced in 7 days; did any human mathematician actually read the entire 166-page document in full within that time period? It seems unlikely. It is probable that OpenAI operates by running large agent harnesses on unsolved problems in the mathematical literature, ranked by sociological 'importance'; they then only publish the positive results. This is highly deceitful. It moreover appears possible that OpenAI's models may resurface mathematical ideas from private user data (in the form of chat logs), raising severe data privacy concerns, as well as issues of plagiarism, attribution, and credit.
By obscuring the methodology of these results, OpenAI engages in severe research misconduct. If OpenAI insists on once again entering the research domain, they must disclose everything: the methodology and agent harnesses employed; the degree of assistance received from human mathematicians, who may have signed NDAs; a list of all problems which were attempted and the success rate; the Lean certificates which actually convinced OpenAI employees that the results are true; any text data generated by LLMs along the way; the size of the relevant AI swarms; and the financial cost of these results. They should not release post hoc deformalizations of Lean proofs which were not actually created in the research process. Releasing these 100+ problems on any other terms constitutes unethical proof laundering, designed to inflate OpenAI's commercial prospects. If instead (as I suspect) OpenAI will only operate on fraudulent and deceitful terms, then better that these mathematical results are not released at all.
Thank you, AGMAI, for your work.
Vesna Stojanoska, University of Illinois Urbana-Champaign:
I think it is important to step back from the specific question you are posing here (how to release and disseminate results) to better frame the context of the situation we are finding ourselves in. Specifically, we should be asking how these results were obtained, and at what cost.
If a human mathematician secretly got a hold of a colleague’s work in progress, perhaps modified it, and worked really fast to post or publish before the original work appeared, I would like to believe that the community would not be rewarding this mathematician’s misconduct by praising the content of the questionable work, publishing their work in reputable journals, and so on. But the fact is, we have limited infrastructure to deal with this kind of misconduct, and we have relied on the potential for reproach as a preventive measure. Now we are facing the same issue with OpenAI, except it is many orders of magnitude larger, and there is a huge power difference. We can reproach OpenAI all we want, but the best outcome we could hope for might be a slight decrease in their stock value. We need to insist that AI companies transparently provide the precise input on which their LLMs base their outputs, and I would like this to be a matter of course, not only a special requirement in cases of solutions to prominent mathematical problems. This should be a regulatory issue that affects all LLM outputs in all domains of use, not just in math. If this is not something technically feasible at the moment, I believe LLM tools should be recalled and not be used in any capacity until the issue is solved.
We need to insist on full transparency of physical costs such as energy and water spent on training and running LLMs that solve mathematical problems. Again, I think this is a broader issue, and I would like to see transparency regulations that require AI companies to disclose environmental costs for all of their functions, from AI-powered internet searches, to solving open conjectures in math.
Alexander Elzenaar, Monash University:
These companies claim to "build AI in service of humanity" (https://openai.com/index/built-to-benefit-everyone-our-plan/), "not to decide which scientific problems deserve attention or try to solve them all ourselves" (https://openai.com/index/chatgpt-for-academic-researchers/) and "empower scientists and mathematicians with tools that accelerate discovery" (https://openai.com/index/ten-advances-in-mathematics/). So why are they spending so much real money trying to out-run mathematicians (for instance, trying to outcompete postdocs like Julia Stadlmann), rather than simply providing tools and letting mathematicians use them? The answer is simple: these companies are transparently lying about their motivations. Mathematicians are a small global market, and they see us only as a source of advertising wins.
The resulting announcement of the form that 100 `open problems' have been `solved', without giving any specific details, is very hurtful to the standing of mathematics as a profession. It feeds the narrative that mathematics as a field consists of a list of problems which must be solved, and then once we have done so we can all go home. This narrative ignores two facts:
(i) Even mathematicians who primarily work on short and easy to state problems want to understand deeply why the thing is true or false. Communicating this understanding (as well as describing the motivation for, and choosing interesting consequences of, the problems) is a creative process.
(ii) Many mathematicians do not work on easy-to-state problems, and instead spend most of their time navigating and understanding a broad space of facts which may be individually easy
to prove. Trying to mechanise this ignores that the entire point of their work is to produce a human perspective on the landscape, not a list of things which are true and a certificate of truth.
Anyone who has ever tried to explain what they do to a non-mathematician knows that most people think that the job of mathematicians is to mechanically solve a list of problems given to them by someone. Companies with massive political power reinforcing this view, and pushing a message that mathematics can now be done entirely by machines, will only lead to further reduction in funding and job opportunities for young people.