Tag: machine learning

Categories
AI

The New Newton

“Machine learning is a very important branch of the theory of computation… it has enormous power to do certain things, and we don’t understand why or how.”
— Avi Wigderson, Herbert H. Maass Professor, School of Mathematics.

There is a specific kind of silence that permeates the woods surrounding the Institute for Advanced Study (IAS) in Princeton. It is a silence designed for “blue-sky” thinking, the kind that allowed Einstein to ponder relativity and Gödel to break logic. For decades, this has been the sanctuary of the slow, deliberate grind of human intellect—chalk dust on slate, long walks, and the solitary pursuit of elegant proofs.

But recently, the tempo in those woods has changed.

We are witnessing a profound shift in the architecture of discovery. In closed-door meetings and public workshops, the conversation among the world’s top theorists is moving from skepticism to a startled accelerationism. The consensus emerging is that Artificial Intelligence is no longer merely a peripheral calculator; it is becoming an “autonomous researcher.”

The 90% Shift

Some physicists now suggest that AI can handle up to 90% of the routine analytical and coding “heavy lifting” of science. This is a staggering metric. It frees the human mind from the drudgery of calculation, but it also introduces a tension that strikes at the heart of the scientific method. We are moving into a realm where the tool may soon outpace the master’s understanding.

There is a growing realization that we are approaching a horizon where AI finds solutions—patterns in the noise of the universe—that work perfectly but remain mathematically “magic.” We might cure a disease or solve a fusion equation without understanding the why behind the how.

A New Natural Phenomenon

This brings us to a fascinating historical rhyme. Scholar Sanjeev Arora has compared our current moment in AI to physics in the era of Isaac Newton. When Newton watched the apple fall, he could describe the gravity, but he couldn’t explain the fundamental mechanism of why it existed.

Today, scholars at the IAS are looking at deep learning in the same way. They are observing a new natural phenomenon—a digital physics. They are trying to find the “laws” of deep learning, asking why these massive models work when classical statistics suggests they should fail (such as in cases of overfitting).

We are building a new machine, and now we must retroactively discover the physics that governs it.

Steering the Black Box

This is not just a mathematical challenge; it is a societal one. The IAS has wisely expanded this inquiry to the School of Social Science. If we are handing over the keys of discovery to a “black box,” we must ensure we are steering it “for the Public Good.” The distinction between genuine problem-solving—like protein folding—and “AI Snake Oil” in social prediction is vital. We cannot let the magic of the tool blind us to the morality of its application.

The future of science, it seems, will not just be about the genius on the chalkboard. It will be about the partnership between the human question and the digital answer. The challenge for the modern scholar is no longer just to calculate, but to comprehend the alien intelligence we have invited into the library.

Categories
AI AI: Large Language Models

The Allure of Large Language Models: A Personal Connection

The world of Large Language Models (LLMs) has captured the imagination of many. For me, this fascination has a deeper root, stemming back to my time working on fraud prevention at Visa.

Card fraud is an ongoing battle. Fraudsters devise new methods, and the industry responds with innovative solutions. One such threat was the counterfeiting of magnetic stripes on cards. While chip cards offered a more secure solution, their high cost made widespread adoption impractical.

In search of a cost-effective solution, we explored two approaches. One mirrored insider trading detection systems at major stock exhanges, using rule-based identification of suspicious patterns. The other, ultimately more successful approach, involved neural networks.

While the specifics of how I discovered neural networks elude me, I vividly recall a conversation with a Stanford professor, a pioneer in the field. His encouragement spurred us to pursue this technology. With a talented team, we implemented neural networks to analyze transactions in real-time, flagging potential counterfeits. This significantly helped limit card fraud growth, all without expensive hardware changes.

Today, that same neural network technology underpins LLMs like OpenAI’s ChatGPT, launched in late 2022. Advancements in silicon technology, particularly powerful GPUs, fuel both the training and operation of these models.

Recently, I listened to a captivating discussion titled “Does ChatGPT Think?” featuring Stephen Wolfram. That conversation triggered me writing this blog post.

Wolfram’s description of LLMs resonated deeply with me:

“So the big achievement and the big surprise is that we can have a system that fluently produces and understands human language… It’s not obvious that it would work, and it’s a kind of scientific discovery that it’s possible to have a thing like ChatGPT that can produce this thing that’s one of our sort of prize features – namely human language.”

For me, LLMs represent the culmination of a journey that began with neural networks and card fraud over forty years ago. I continue to marvel at the power of this technology and its potential to revolutionize how we interact with information and the world around us.