Author: Scott Loftesness

Categories
AI ChatGPT

AI Chatbots are the new attention getters

The wave of AI chatbots continues to build – with the recent public release of ChatAPT, the new Poe app from Quora, today’s Bing announcement from Microsoft and whatever might be coming from Google, Facebook, Apple and others.

I’ve been doing quite a bit of experimenting with ChatAPT in particular – as you can see from some of my recent posts. Today I’ve been spending time with Poe on my iPhone – it’s a really well done app that makes it even easier to interact conversationally with AI chatbots (Poe supports multiple chatbots).

One thing is clear from my play – these new “tools” are true “attention sinks.” As you ask questions and research a topic, the ability to interact and dive deeper is quite addicting – and time consuming! It’s all an interesting experience.

With the potential for significant monetization for companies that can captivate our attention through their apps, the time we spend with AI chatbots may come at the expense of time spent on other apps, which could impact the ability those other apps to continue to monetize at the same level. This shift will be intriguing to observe.

Categories
iPhone 14 Pro Max Menlo Park

A Rainy Walk in the Park

Yesterday we had some periods of rain – including a brief spattering of tiny hail. During a break in the rain, I headed out on my morning walk – only to have rain move in again by the time I reached the park. I enjoy walking in the rain – if it’s not too cold and not too windy – so I enjoyed it.

I wanted to play around a bit with Cinematic Mode on my iPhone 14 Pro Max. I made several movies – a couple of them included below. It is interested to see how the water and the splashes from the raindrops made focusing challenging for Cinematic Mode. I didn’t override anything manually – these two videos are straight out of the camera.

Sharon Park in the Rain #1
Sharon Park in the Rain #2
Categories
AI AI: Diffusion Models ChatGPT

Diffusion Models and Image Creation

I’ve been doing some more exploring with ChatGPT – this time asking it to educate me on how diffusion models work – they’re the underlying technology behind apps like DALL-E, Stable Diffusion, Midjourney, etc. Below is what I learned during my conversation with ChatGPT. The image below was created using DiffusionBee, an app based on Stable Diffusion. I used the input: a colorful illustration of how diffusion models work.

Continue reading “Diffusion Models and Image Creation”
Categories
AI AI: Large Language Models AI: Transformers

Large Language Models and Transformers

This morning I noticed an article titled “Understanding AI” in the Financial Times about an upcoming two-week series on artificial intelligence. That prompted me to just ask ChatGPT to write a 1,000 word blog post describing large language models (LLM) and transformers. Below is what I got back from ChatGPT…a pretty useful description of both! By the way, today’s Wall St. Journal has an article titled “Sports Illustrated Publisher Uses AI to Write Articles” that discusses how some publications are beginning to use AI chat bots to generate stories for publication.

Also earlier this week I watched a YouTube video from Stanford with Professor Christopher Potts titled GPT-3 and Beyond – a bit technical for me but he provided a fascinating introduction to the progress in AI.

Large Language Models and Transformers

Large language models and transformers are artificial intelligence technologies that have been revolutionizing the field of natural language processing. These models are capable of performing complex language tasks such as text classification, question answering, and machine translation. In this post, we will provide a comprehensive overview of large language models and transformers, their history, how they work, and their current and future applications.

A brief history of language models

The study of language models has a long history dating back to the 1950s when the first attempts to build machine translation systems were made. At that time, the focus was on building rule-based systems that relied on pre-defined grammar and syntax rules to translate between languages. However, these systems were limited in their ability to handle ambiguity and variability in natural language.

In the late 1980s, the field of machine learning emerged, and language models started to use statistical techniques to improve their performance. Over time, researchers have developed more sophisticated models that can learn from large amounts of data and achieve better performance on various language tasks.

Introduction to large language models

Large language models are deep neural networks that are trained on massive amounts of text data to perform a wide range of natural language processing tasks. The key innovation behind these models is the ability to process and understand the context of words and phrases in a sentence. This allows them to generate more accurate predictions and perform more complex language tasks than traditional rule-based systems.

Large language models are trained on billions of words from a vast variety of text sources, including books, news articles, websites, and social media. This enables them to learn the patterns and relationships between words and sentences, as well as the meaning of words and phrases in different contexts.

Introduction to transformers

Transformers are a type of large language model that have been specifically designed for natural language processing. They were introduced in 2017 by Vaswani et al. in a paper called “Attention is All You Need.” The key innovation behind transformers is the attention mechanism, which allows the model to focus on specific parts of the input sequence when making predictions.

Traditional language models process sequential data by breaking it down into smaller pieces and processing each piece in a linear fashion. This makes it difficult for the model to consider the relationships between words that are far apart in the input sequence. Transformers, on the other hand, use attention mechanisms to allow the model to weigh the importance of different parts of the input sequence when making predictions.

How transformers work

Transformers work by first encoding the input sequence into a set of continuous vectors. These vectors are then processed by a number of self-attention layers, which use the attention mechanism to focus on specific parts of the input sequence when making predictions.

The attention mechanism in transformers works by calculating the relationships between all pairs of words in the input sequence. The model then uses these relationships to weigh the importance of different parts of the input sequence when making predictions. This allows the model to consider the context of words and phrases in the input sequence, leading to more accurate predictions.

After the self-attention layers, the encoded vectors are processed by a feed-forward neural network to make the final predictions. The feed-forward network includes a number of dense layers, which are fully connected layers that allow the model to make more complex predictions.

Applications of large language models and transformers

Large language models and transformers have a wide range of applications in the field of natural language processing. Some of the most notable applications include:

  • Text classification: Large language models can be used to classify text into different categories, such as sentiment analysis, topic classification, and spam detection.
  • Question answering: Large language models can be used to answer questions by extracting relevant information from a text corpus and providing a concise answer.
  • Machine translation: Transformers have been shown to be highly effective for machine translation, as they are able to handle the complexity of translating between languages and capture the relationships between words and phrases in different languages.
  • Summarization: Large language models can be used to summarize long documents into shorter versions that retain the main information.
  • Dialogue systems: Large language models can be used to build dialogue systems that can understand and respond to human language in a natural way.
  • Generative models: Large language models can also be used for generative tasks, such as writing creative fiction or composing music.

Current and future developments

In recent years, there has been rapid progress in the development of large language models and transformers. This is due in part to the availability of massive amounts of text data and the advancement of computational resources, which have allowed researchers to train models that are larger and more complex than ever before.

One of the current challenges in the field is reducing the computational cost of training and using large language models. This is important because large language models require vast amounts of computational resources, and their deployment is limited by the availability of these resources.

There are also ongoing efforts to improve the ethical and social implications of large language models and transformers. For example, there is concern about the potential for these models to reinforce biases and perpetuate harmful stereotypes. Researchers are working to develop methods for mitigating these risks and ensuring that large language models are used in a responsible and ethical manner.

Conclusion

Large language models and transformers have been making a significant impact on the field of natural language processing. Their ability to process and understand the context of words and phrases in a sentence has led to advances in a wide range of applications, including text classification, question answering, and machine translation. As these models continue to evolve, we can expect to see even more exciting developments in the future.

Categories
Apple General Magic Innovation

Magic in the Air

I’m not sure what provoked my interest earlier this week, but I enjoyed watching the documentary from a few years ago on the history of General Magic. Living in Silicon Valley, I remember hearing stories about the company – and the talented people working at the company led by CEO Marc Porat.

Watching the documentary provided some great background into how the company innovated in so many ways yet wasn’t able to be a commercial success. Lessons learned included how great ideas fail when they’re too early for the technology to support them, the implications of trying to partner with other large companies, and the dangers of competition that “fast follows” on your ideas.

Last night, I stumbled across another fun video produced by the Computer History Museum about the history of MacPaint, Quickdraw, and more as told by Bill Atkinson and Andy Hertzfeld. They were both early employees at Apple, very important software developers on the Mac platform, who left Apple to be early founders of General Magic. Listening to them talk about the early days of development at Apple was very interesting.

Categories
Aviation Travel

Ode to a Queen

A few days ago, Boeing rolled out the last production model of its 747 line of wide body aircraft. Wikipedia noted:

The first flight took place on February 9, 1969, and the 747 was certified in December of that year. It entered service with Pan Am on January 22, 1970. … The final 747 was delivered in January 2023 after a 54-year production run, with 1,574 aircraft built.

I have many fond memories of this great airplane – from seeing it for the very first time flying in to land at San Francisco International Airport (which must have been in 1970) to my first flight which – best of my recollection – was on TWA flying from San Francisco to New York. My last 747 flight was on a Lufthansa 747-8 flying out of Bangalore to Frankfurt in 2016. 

I had a few flights in a seat on the 747’s upper deck – a special treat! The upper deck was also where the cockpit was located – it always seemed to be so high up off the runway that landing a 747 seemed to require some special skill and depth perception! On the other hand, the airplane’s landing gear really smoothed out landings as it had this swing mechanism where the rear wheels on the main landing great touched down first and then pivoted to smoothly bring down the front wheels. The early models of the 747 had a circular staircase to the upper deck where there was a lounge instead of seating. This upper deck cockpit design also facilitated adding a nose door which pivoting upward in the freighter version of the 747. In fact, the last 747 delivered was a freighter to Atlas Air.

Speaking of Atlas Air and that last 747, after delivery it flew from Seattle to Cincinnati as it was put into service. The pilots on the first flight of that last 747 has some fun on their flight – trading a lovely tribute to the “Queen of the Skies” on their radar track.

NASA used a special version of the 747 as the transporter for the space shuttle. In 2012, NASA flew a final flight of its 747 carrying the shuttle Endeavor to its new home in a museum in Los Angeles. Along the way, the NASA 747 toured the San Francisco Bay Area and I was fortunate to be able to make a few images of that flight:

While the last 747 has been delivered by Boeing, it will continue to be flown for many years ahead. Most airlines have removed 747s from their fleets but a few (including Lufthansa) continue flying them. I’d enjoy taking another trip on one of these beautiful aircraft. In the meantime, I’ve got lots of good memories about trips and sightings of the 747.

Update: February 03, 2023 – a few additional thoughts on the 747.

  • For many years, two 747’s have provided air transport for the US presidents – and the current 747’s are soon (?) to be replaced by two new models that have been undergoing customization for some time.
  • Wired added an article about the negatives of the 747 – saying it should have been retired “many years ago”. The article also adds more interesting details about the history of the 747. Fuel economy is the biggest detractor: “A Boeing 747-400, which was manufactured between 1989 and 2009, costs around $26,635 an hour to run. A Boeing 787-8, which is still produced today, costs $14,465 an hour to operate—45 percent cheaper.”

Update: February 04, 2023 – The New Yorker also has an new article about the 747 titled “The World the 747 Didn’t Predict“.

Because the 747 could now seat more travellers on a single flight, airlines were able to sell more tickets at lower prices. Suddenly, travel, particularly intercontinental travel, was accessible to people who had rarely, if ever, been in the air. The 747, in a sense, taught the world to fly.

Categories
Amazon.com

What’s love got to do with it?

Like many of you I’m sure, for many, many years my go-to purchasing behavior has started with doing a search on Amazon to look for whatever it is I might be needing to buy. Rather than wasting time trying to shop locally or doing online searches for items, it was just easier and simpler to do that quick Amazon search for an item.

With Prime, Amazon had biased my behavior in favor of doing a quick purchase from them rather than looking elsewhere. I had confidence that I was getting a fair price and that the item would be in my hands within a day or two. Maybe it’s too strong a word for something as mundane as shopping, but I did love the experience of buying from Amazon – because it was so simple an easy.

But that’s changed – slowly over time – and I’m not the only one who’s noticed it. A couple of recent stories (among others) highlight this:

  • New York Magazine: The junkification of Amazon – “Late last year, The Wall Street Journal reported that Amazon’s customer satisfaction had fallen sharply in a range of recent surveys, which cited COVID-related delivery interruptions but also poor search results and “low-quality” items. More products are junk. The interface itself is full of junk. The various systems on which customers depend (reviews, search results, recommendations) feel like junk.”
  • Barry Ritholtz: How Amazon became ordinary – “Generally speaking, I have been a satisfied Amazon consumer – at least up until the pandemic. That was where cracks in the Amazon armor began to show.”

This unfortunate evolution in how Amazon feels to me, a long-time satisfied customer, makes me sad and disappointed. Clearly, I’m not alone. I wish the senior management team at Amazon would sit up and pay better attention to the impacts their “optimization efforts” have been having on the satisfaction of long-time customers. They must understand that an erosion of satisfaction leads directly to switching behavior.

More recently, Amazon announced the elimination of another benefit (Amazon Smile) which, in the overall scheme of things, wasn’t a big deal – but which did make me feel better about my spending at Amazon. Amazon Smile benefited a local non-profit by donating 0.5% of my spending on Amazon at no additional cost to me. This really feels like a “penny wise, pound foolish” move – one that doesn’t lift up my opinion of the company. Another brand diminishing move.

Meanwhile, this morning an Amazon blog post trumpeted how it’s ranked as a widely admired company in a recent Fortune survey: “Amazon ranks No. 2 for the seventh year in a row.”

There is so much to admire about Amazon and everything it provides. Yet it feels to me like it’s slipping and leaving me disappointed. I’d like to get back to feeling delighted with the Amazon shopping experience instead.

Let’s hope the company’s management realizes I’m not alone.

Categories
AI ChatGPT

The Ghost in the Box

I happened to hear a discussion on a YouTube video yesterday about Niklas Luhmann’s mention of how his “zettlekasten” seemed to act like a “ghost in the box”. I did a quick Google search for that phrase but didn’t find anything particularly helpful.

Earlier I had watched Nick Milo’s video about how he’s been using OpenAI’s ChatGPT for research and other things. He talks about a near future where he expects to have it nearby whenever he’s working on his computer.

So I decided to ask Chat GPT to help me understand more about the “ghost in a box” by writing a 1,000 word article about it. Here’s what I got back from ChatGPT:

The Ghost in the Box

The concept of “ghost in the machine” was popularized by philosopher Gilbert Ryle in his 1949 book “The Concept of Mind.” The phrase refers to the idea that human consciousness and mental states cannot be reduced to mere physical processes in the brain.

German sociologist and systems theorist Niklas Luhmann took this concept a step further in his theory of social systems. He proposed that society itself can be seen as a “ghost in the box,” a complex system that is made up of smaller, interconnected parts that cannot be fully understood or controlled by any single individual or group.

According to Luhmann, society is made up of various “function systems,” such as the legal system, the economic system, and the political system. These systems operate independently of one another, but are also interconnected and dependent on each other. Each system has its own “code,” or set of rules and norms that govern its operation.

Luhmann argues that society as a whole can be seen as a “communication system,” which is constantly producing and processing information. This information is then used by the various function systems to make decisions and take action. However, the complexity and sheer volume of this information make it impossible for any one individual or group to fully understand or control the system.

This idea of society as a “ghost in the box” has important implications for how we understand and interact with the world around us. For example, it suggests that we can never fully understand the causes of social phenomena or predict their outcomes. It also implies that we should be cautious about attempts to manipulate or control the system, as these efforts are likely to have unintended consequences.

Furthermore, Luhmann’s theory also highlights the importance of communication and understanding the code of the different function system in order to function in the society. He also emphasizes on the importance of social differentiation, as it allows different function system to operate independently while still being interconnected.

In conclusion, Luhmann’s concept of society as a “ghost in the box” offers a unique perspective on the complexity and unpredictability of social systems. It reminds us that we can never fully understand or control the world around us, and that our actions and decisions are likely to have unintended consequences. Thus, it is important to be aware of the limitations of our understanding, and to approach the world with humility and caution.

Note: This was generated using ChatGPT Jan 9 Version.

Categories
Living Tracy Loftesness

Back to a Better Future

Half Moon Bay – 2006

Last year, in my first blog post titled “Life is a Contact Sheet“, I said:

Happy New Year! Let’s work towards better outcomes in 2022 than we had in 2021! Like most I’m looking forward to leaving 2021 behind and excited about what the future could bring! Now onward to my first post of this new year!

While I was excited for what 2022 might bring, I now look back on this past year with much sadness – as the hopes of me and many others just didn’t happen and way too much sadness and gloom came into our world instead.

Late last month, our daughter Tracy passed away – the victim of a very aggressive cancer that was identified only weeks before and which failed to yield to treatment. Tracy was the true adventurer in our family – and her spirit lives on to inspire us towards a better future. We have really appreciated all of the kind words and shared memories from so many who knew Tracy.

New Year’s Eve each year is our Dad’s birthday – bringing back so many memories of our times with him. 2022 was his 101st birthday. He also loved Tracy very much.

Of course there we some bright spots in 2022 too – although it’s difficult at times to remember those and to keep them in perspective. The challenge – and ultimately our reward, of course – is to remember that life is full of these ups and downs.

So here’s hoping for a better future in 2023 – we’ll hug our loved ones tighter, try to do a better job being good friends, and savor those good moments and memories when they come along. Onward!

In November 2010, Tracy and I took a cooking class at Village Pub in Woodside – learning how to cook a Thanksgiving feast. We had such a good time that day – a great memory.

Categories
Books Future Quotations

Thinking about the future…

Virtually everyone thinks in first person when they imagine their recent past, present, or near future. Likewise, almost everyone switches to third person when they think about their far past or far future, usually defined in the scientific literature as ten years in either direction from today. This shift in mental perspective is why you can often look back at emotionally charged moments in your life, after enough time has passed, and see things from a more detached, clearer point of view. Your brain is literally processing them from a more insightful vantage point. Likewise, this is why taking a mental time trip ten years to the future can help you feel “unstuck” emotionally. You momentarily get a break from your normal mode of thinking and feeling and get to float above it all, like a satellite looking down from space.

Jane McGinigal – Imaginable