Zima Red ep 144: Bilawal Sidhu - How AI is changing creative technology..and everything else
Primer: As we speak, AI is revolutionizing our world. How will it impact creative content creation and the Metaverse? What are the pitfalls we should pay attention to? Let’s find out from Bilawal Sidhu in this episode of Zima Red.
Background
A creative technologist, product builder, and content creator
At 12 years old, he taught himself 3ds Max, Maya, etc.
Studied Computer Science and Business at the University of Southern California
Spent a decade in tech across AR, VR, and 3D mapping
What Is Creative Technology?
Art and science of visual effects, 3D animation, and augmented reality
Impacts both the creation and consumption side — How you create content and engage and consume content
Involves multiple disciplines like computer vision, computer graphics, art, and storytelling
The Metaverse
What Is It?
A spatial embodiment of the internet we all know and love
It is not a 2D document-object model that is connected together with hyperlinks
AR and VR are not required. You can access the Metaverse on a 2D screen
Metaverse: From Obscurity To Cringe
The term metaverse had its moment under the sun last year
The hype started in 2021
Hype cycles are getting compressed
Because of the pandemic, these trends got accelerated
Will take some time for the devices to catch on
The Evolution Of The Metaverse
A mainstream moment, like an Apple XR/AR headset, would boost things
A lot of hard technical problems need to be solved
Generative AI will be crucial to populating the metaverse
The barrier to creating 3D content is very high
Think that we will see incremental use cases
“We'll see layer by layer, these capabilities stack on top of each other until one day, you and I are in a fully virtual embodied experience.”
- Bilawal Sidhu
The missing piece is interoperability
Has The Internet Made The World A Better Place?
It’s a double-edged sword
Don’t think the AI revolution could have happened without the internet
Will The Metaverse Be A Force For Good?
Opportunity for 2 kinds of future (utopia or dystopia)
Hopes that the Metaverse affords a middle path
There’s potential for it to be less taxing on us (viewing things on our phone)
Product builders, platforms, and governments are increasingly going to play a larger role in setting the right incentives for the ecosystem
Invasive VS Non-Invasive Brain-Machine Interface (BMI)
It’s not just Sci-Fi
It has been around spiritual and Eastern esoteric traditions for thousands of years
Elon Musk said we are in the base layer of the simulation
Some people can lucid dream and control what happens
Shamans and gurus can control their parasympathetic functions
Think that we will lean into computing capabilities that we have in our brain to create more predictable and controlled experiences
There will be dystopian monetization opportunities and also utopian applications
People who have sensory impairments will be the first in line to benefit from this technology
The Future Of Entertainment
Entertainment involves a small number of studios creating one-size-fits-all content that is centralized around Hollywood
YouTube was the next step on the trajectory:
1% of total users are creators
Creation takes effort and creators come up with content that would never get greenlit by the formal studio world
MrBeast is one example
“We're now transforming to this sort of creator, business entrepreneur, creatorpreneur model where they're in charge of their own content. They're building their own audiences, and then they're creating products and services around that.”
- Bilawal Sidhu
The TikTok era further democratized content creation:
30-50% of the user base is creating content
YouTube shorts and Instagram reels emerged
The barrier to content creation keeps going lower
Instead of Adobe Premier and DSLRs, people are now just using their phones
AI is disrupting content creation even further
Shift from social graph-based discovery mechanism to interest graph-based discovery
Personalization Of Content For Creators And Consumers
Creators have a choice to make — Whether they want to provide reinforcement learning to replace themselves
Cloud-based creation tool companies could mine insights from content creators that use their tools and replace them completely
Virtual influencers have been a massive trend in Japan since 2016
It could go in a dystopian direction where Hollywood makes trophies out of talent (e.g. bringing Marilyn Monroe back from the dead)
Believes more in keeping creators at the center of the creation experience
Seeing a lot of backlash from artists
Humans are very good at conjuring up the next more complex medium that we couldn't possibly have created foR
People get bored of mediums easily — even podcasts, which are reinvented radio
When reinvention happens, you will want that X factor that makes you feel alive
Extreme Form Of Personalization
Don’t think that algorithms and machines are inherently good or evil
The stakes are like nuclear because of the ramifications
Every religious and spiritual tradition talks about envy
The ability to peer into a perfectly curated picture of someone’s life on social media is not good for one’s mental health
On the plus side, social media helps you to connect with your friends
If you outsource your discriminator to a social media feed, you become more susceptible to entities that are using advertising platforms to influence you
Maintaining An Advantage In An AI World
Have a friend who is doing an AI startup in the past batch of YC
The vast majority of companies are essentially wrappers around OpenAI
Thinks that if these companies are not able to make money within the next 2 years, they will end up failing
Jim from Nvidia wrote a tweet thread where he suggested that upscaling an image on Midjourney could be used as a cue as to what is aesthetically pleasing. This could be used in a feedback loop to make their model better
A moat could be built around a compelling product user experience
His view is that people should find domains that are interesting, have a clear ROI, and go solve those problems for people
A lot of companies in the AR/VR space made money by doing boring things like stock picking inside warehouses
At their best, algorithms should be reflections of user preferences. However, people believe that there are other forces at play (CCP putting out disinfo on TikTok)
Creators are building their own islands of influence
The incremental cost of creating digital products, podcasts, and software is trending toward zero. Will see more creators building their communities
Bringing The Metaverse To Life
Involves blending the physical and digital worlds together
Technologies such as Photogrammetry and Neural Radiance Fields (NeRF) are new ways of doing reality capture
Now Generative AI is able to kitbash reality
You can remap the world once every few years/major cities once a year — capture 50 megapixels of imagery and stitch them into a 3D model
“Reality capture got democratized because sensors got better, compute got cheaper.”
- Bilawal Sidhu
Instead of having artists build things from scratch, they kitbash reality and mash things together for Triple-A game titles
You can digitize your house for the memories or even send it to the contractor that’s going to redesign your place
A kid with a drone, reality capture, and Unreal engine could recreate GTA in their hometown
Making Money From Reality Capture
Centralized large players are making the most money
The tech has only been democratized in the last few years
VCs have reached out to him about NeRF companies/start-ups
Is not convinced that NeRF is a business. It’s more like a feature that is part of a larger product
Neural Radiance Fields (NeRF)
Reality capture can be divided into 3 categories:
Visualization
Analysis
Machine understanding/localization
Visualization involves creating a human-readable 3D model of the world
The techniques for visualization have to do with photogrammetry — The art and science of measuring the world using imagery observations and sensor data
A 3D mesh consists of a triangulated mesh with textures on it
Reality is complex — There are reflections, refractions, material properties, etc. that are difficult to represent with 3D meshes
There’s also the problem of View Synthesis — Given a bag of images, how do you synthesize an intermediate viewpoint across these images
Start-up founders are trying to figure out the monetary use cases:
The VFX industry is in a spiral, so there’s only a handful of customers
For real estate, there’s Matterport
If you try to map everything in reality, it’s not possible to take all the photos:
NeRFs degrade more gracefully than photogrammetry
Photogrammetry would produce blobs/holes in the images
Diffusion models are becoming popular for solving reality capture
On Large Language Models (LLMs)
There’s a hot debate as to whether is it a reasoning engine
Thinks that Artificial General Intelligence (AGI) is too much power for everyone to wield
Thinks that synthetic media is a solvable problem
Multimodal models (like GPT-4) should not be open-sourced as the abuse potential is far too high
Open-source maximalists have told him that a vast amount of people are good actors and that they can keep the bad actors in check. He does not buy this explanation
Sees a future of a blooming ecosystem — A number of large models and a bunch of small ones that work at the edge
Regulating LLMs
A huge debate that is playing out right now
The challenge is do people even need more than the public internet?
After the crash in GPU prices after the crypto bust, researchers started scraping images off the internet
There’s also the debate about whether you need specialized data or just open-source data would do
China has already issued synthetic media guidelines 6 months ago
Will be harder to regulate these models in the West
Different types of AI capabilities need to be regulated differently — Not as concerned about content generation models as compared to multimodal models
Tech And The Defense Industry
A lot of people in tech have lost track of the fact that tech would not have existed if not for the defense establishment
The Internet of Things (IoT) was made for the Vietnam war
It applies to early research in computing/internet as well
We need to return to the past when tech companies play a principal role in defense
People criticized Palmer Luckey for his involvement in the defense industry
Tech people have to think about the public and private applications of AI and the good and the bad
Predictions About AI
Hope that Apple’s Worldwide Developers Conference (WWDC) happens and that they show that:
Stable Diffusion could be run on 30 frames per second on their new M series chip
People can run an LLM on a $3000 laptop
Argues that people are already dating AI on character.ai based on the session lengths
Millennials and Gen Z will have AI therapists
Independent creators will be able to chain AI models together to produce studio-level output
Which Industry Will Be Most Impacted by AI Soonest?
Any industry that falls under the bucket of knowledge work
“I think almost every industry that deals with digital tools and involves chaining them together to offer products and services are going to be absolutely disrupted, upended.”
- Bilawal Sidhu
Capabilities that were reserved for specialists will become available to the masses
When Do We Get AGI?
Depends on how we define AGI
If you define AGI as everything a human can do, it could be in 5-10 years
Best Advice He Has Received
Be a perpetual optimist and have a relentless desire to learn
Have to learn in a sustainable fashion
The Traits That Most Define Him
Adaptability
Reinvents himself every few years
What Motivates Him
Talking to cool people, working with cool people, and making cool stuff
Democratizing creativity and blending reality and imagination together
All information presented above is for educational purposes only and should not be taken as investment advice. Summaries are prepared by The Reading Ape. While reasonable efforts are made to provide accurate content, any errors in interpreting and summarizing the source material are ours alone. We disclaim any liability associated with the use of our content.