OpenAI’s DevDay updates revealed

Welcome, AI enthusiasts.

OpenAI’s DevDay could have skipped the spectacle this time with no live stream — but we caught the event live and secured exclusive details on recent releases.

With 4 recent major developer-focused announcements, and a personal Rundown Q&A with OpenAI’s Head of Product, we’ve got an enormous one today. Let’s get into it…

In today’s AI rundown:

• OpenAI makes 4 major announcements at DevDay

• Microsoft Copilot gets voice, vision upgrade

• Exclusive DevDay Q&A with OpenAI’s Olivier Godement

• Extend images at no cost with HuggingFace

• 5 recent AI tools & 4 recent AI jobs

• More AI & tech news

Read time: 4 minutes

LATEST DEVELOPMENTS

OPENAI

⚙️ OpenAI makes 4 major announcements at DevDay

The Rundown: OpenAI just held its DevDay 2024 event, unveiling a collection of recent API features and enhancements designed to make its AI systems more accessible, efficient, and cost-effective for developers to construct with.

The main points:

• Realtime API enables speech-to-speech application constructing using the identical model that powers Advanced Voice, with the flexibility to pick from six voices.

• Model Distillation simplifies fine-tuning smaller models using outputs from larger ones, making training more accessible to developers.

• Prompt Caching reduces costs by nearly 50% across models and quickens responses by as much as 80% when reusing recent input tokens in API calls.

• Latest Vision Nice-Tuning allows models to be trained with each images and text, allowing developers to optimize tasks like image recognition and evaluation.

Why it matters: While this yr’s DevDay could have lacked the normal hype of a typical OpenAI event, the releases are still set to have an amazing impact. These API updates not only enable the creation of entirely recent, exciting experiences but additionally lower the barrier to entry, for builders across OpenAI’s platform.

TOGETHER WITH SYNTHFLOW

🗣️ AI phone calls that sound human

The Rundown: Synthflow’s AI-powered phone calls enable interactions which might be indistinguishable from human conversations — revolutionizing the way in which businesses handle customer support.

With Synthflow, you may:

• Create lifelike AI voices that talk naturally in multiple languages

• Design custom conversation flows to handle various scenarios

• Integrate seamlessly along with your existing systems for efficient call handling

• Scale your customer support without compromising on quality

Try Synthflow today and experience the longer term of customer communication.

MICROSOFT

🚀 Microsoft Copilot gets voice, vision upgrade

The Rundown: Microsoft just announced a slew of AI upgrades coming to its Copilot assistant for Windows PCs, including recent vision and voice capabilities, personalization enhancements, a re-release of the controversial Recall feature, and more.

The main points:

• Copilot Voice allows users to interact with natural speech, adding conversational and intuitive communication much like OpenAI’s Voice Mode.

• Copilot Vision enables the AI to know and interact with web content a user is viewing, offering context-aware help inside the Microsoft Edge browser.

• ‘Think Deeper’ gives Copilot recent enhanced reasoning capabilities using chain-of-thought reasoning powered by OpenAI’s o1 model.

• Microsoft’s ‘Recall’ feature is about to return, requiring an opt-in with upgraded privacy and security measures.

• Microsoft AI CEO Mustafa Suleyman highlighted Copilot’s ability to ultimately ‘act in your behalf’ and adapt to user’s personal preferences and desires.

Why it matters: Microsoft is bringing the warmth with these major Copilot upgrades, levelling up the assistant to align with the most recent cutting-edge AI features across the industry — while bringing users one step closer to a really agentic experience.

OPENAI DEVDAY

🎤 Exclusive DevDay Q&A with OpenAI’s Olivier Godement

The Rundown: We caught up with OpenAI Head of Product Olivier Godement after he led the fundamental keynote at Tuesday’s DevDay event for some exclusive insights on the brand new Realtime API (Godement’s responses are summarized for brevity).

On the Realtime API: Godement says that “Until immediately, voice has been a second activity“, and that the Realtime API goes to make AI significantly more accessible because many individuals in the true world prefer to talk over reading or texting.

On real-world use cases: Godement believes the Realtime API could have a “no-brainer” impact on customer support, education, and training. He also believes there shall be many ‘non-obvious‘ use cases which might be hard to predict now.

On pricing: Converted to seconds, audio input is ~6 cents per minute, and output is ~24 cents per minute. While currently high, Godement confirmed that there are “huge pricing decreases on the roadmap.”

On the Twitter misinterpretation: Godement also mentioned a misinterpretation of pricing after the announcement—when users mentioned how much it costs per hour, they multiplied cost as if the input/output were constant. Nevertheless, every time humans talk, there may be silence—it’s not a continuing flow. The model won’t charge you for silence.

On future modalities: For now, Realtime API only supports text and audio. Nevertheless, Godement believes that image and video are the subsequent milestones on the road to agents that may perceive the world identical to a human. He also mentioned that image and video understanding specifically, will “turbocharge customer support” when the model has the flexibility to know pixels on a screen in real-time.

PRESENTED BY INNOVATING WITH AI

💼 Start your profession as an AI Consultant

The Rundown: Innovating with AI’s recent program, AI Consultancy Project, equips AI enthusiasts with all of the resources to capitalize on the rapidly growing AI consulting market – which is about to 8x to $54.7B by 2032.

This system offers:

• Tools and framework to seek out clients and deliver top-notch services

• A 6-month roadmap to construct a 6-figure AI consulting business

• Student landing their first AI client in as little as 3 days

Click here to request early access to The AI Consultancy Project.

AI TRAINING

🖼️ Extend images at no cost with HuggingFace

The Rundown: Hugging Face’s free AI image outpainting tool allows users to increase their images with custom aspect ratios for various use cases, similar to optimizing images for any social media platform.

Step-by-step:

1. Visit the “diffusers-image-outpaint” Hugging Face space.

2. Upload your image to expand.

3. Set your required aspect ratio and alignment (e.g., 1:1, middle).

4. Adjust advanced settings like output size and input image resize.

5. Click “Generate” and watch AI expand your image!

NEW TOOLS & JOBS

Trending AI Tools

• 🎥 Video SDK 3.0 – Construct and integrate real-time multimodal AI characters

• 📭 Inbox Zero – An open-source, AI personal assistant for email

• 👩🏻‍💻 Graphite – Your AI code review companion

• 📚 Ello – An AI reading companion for youngsters offering personalized support

• 🗣️ VivaChat – FaceTime video chat with realistic AI personas

Latest AI Job Opportunities

• 💼 Palantir Technologies – Mobility Tax Manager

• 📈 Databricks – Business Development Representative

• 🤖 C3 AI – Pre-Sales AI Director

• 🚀 Notable – Solution Delivery Manager

QUICK HITS

OpenAI founding member Durk Kingma announced that he’s joining Anthropic, reuniting with several former OpenAI employees and highlighting the corporate’s mission of responsible AI development in his X post.

Pika Labs unveiled Pika 1.5, a brand new video generation model upgrade featuring enhanced effects, realistic movement, longer clip creation, and cinematic capabilities.

Anyscale unveiled major upgrades to its AI platform at Ray Summit 2024, including a GPU-native Ray architecture, RayTurbo for enhanced performance, Ray Data for unstructured data processing, and more.

U.S. AI chipmaker Cerebras officially filed for an IPO, with the Sam Altman-backed Nvidia competitor expected to be valued at between $7-8B.

Meta released the open-source code and developer suite for its Segment Anything Model (SAM) 2.1, an upgraded version of its image and video segmentation tool.

Nvidia introduced NVLM 1.0, an open-source family of multimodal models that achieve SOTA performance on vision-language and text tasks.

Pinterest launched Performance+, a collection of recent AI tools for advertisers that features the flexibility to create background images for products and automation features for ad campaigns.

THAT’S A WRAP

See you soon,

Rowan, Joey, Zach, and Alvaro—aka The Rundown Team