techcommunity.microsoft.com
Open in
urlscan Pro
2a02:26f0:480:58e::207e
Public Scan
Submitted URL: https://cktxj04.na1.hubspotlinks.com/Ctc/RH+113/cKTxj04/VX5DV51CQQ4-W4LYbnx4B6qJsW46ZLVZ5nQTmqN6rgBpH5kBVqW69t95C6lZ3nKW5WP3GW8SS7c6W...
Effective URL: https://techcommunity.microsoft.com/blog/azure-ai-services-blog/transforming-video-into-value-with-azure-ai-content-understanding/42...
Submission: On November 26 via manual from IN — Scanned from DE
Effective URL: https://techcommunity.microsoft.com/blog/azure-ai-services-blog/transforming-video-into-value-with-azure-ai-content-understanding/42...
Submission: On November 26 via manual from IN — Scanned from DE
Form analysis 1 forms found in the DOM
<form><span
class="fui-Input r1oeeo9n fui-SearchBox HeroBanner_custom-hero-banner-search-box__vhDms ___q5syn70 f1w5jphr fk6fouc fod5ikn figsok6 faaz57k fjuset5 f1eyhf9v f1ng84yb fhxju0i fvcxoqz f1ub3y4t f1l4zc64 f1m52nbi f8vnjqi fz1etlk f1klwx88 f1hc16gm f1xzfw5u"><span
class="fui-Input__contentBefore r1572tok fui-SearchBox__contentBefore ___xfmmu60 f16u2scb"><svg fill="currentColor" class="___12fm75w f1w7gpdv fez10in fg4l7m0" aria-hidden="true" width="1em" height="1em" viewBox="0 0 20 20"
xmlns="http://www.w3.org/2000/svg">
<path d="M8.5 3a5.5 5.5 0 0 1 4.23 9.02l4.12 4.13a.5.5 0 0 1-.63.76l-.07-.06-4.13-4.12A5.5 5.5 0 1 1 8.5 3Zm0 1a4.5 4.5 0 1 0 0 9 4.5 4.5 0 0 0 0-9Z" fill="currentColor"></path>
</svg></span><input type="search" placeholder="Search this community" aria-label="searchPlaceholder" title="Please enter your search term(s) and then press return key to complete a search."
class="fui-Input__input r12stul0 fui-SearchBox__input ___1bonyz8 fk8j09s f11gcy0p f18izjht fcoa6sg" value=""><span
class="fui-Input__contentAfter r1572tok fui-SearchBox__contentAfter ___13sb39p f16u2scb f1cnd47f f1ufnopg fk73vx1 fniina8 f3tsq5r"><span role="button" aria-label="clear" tabindex="-1"
class="fui-SearchBox__dismiss r1pvzcuu ___xfmmu60 f16u2scb"><svg fill="currentColor" class="___12fm75w f1w7gpdv fez10in fg4l7m0" aria-hidden="true" width="1em" height="1em" viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg">
<path
d="m4.09 4.22.06-.07a.5.5 0 0 1 .63-.06l.07.06L10 9.29l5.15-5.14a.5.5 0 0 1 .63-.06l.07.06c.18.17.2.44.06.63l-.06.07L10.71 10l5.14 5.15c.18.17.2.44.06.63l-.06.07a.5.5 0 0 1-.63.06l-.07-.06L10 10.71l-5.15 5.14a.5.5 0 0 1-.63.06l-.07-.06a.5.5 0 0 1-.06-.63l.06-.07L9.29 10 4.15 4.85a.5.5 0 0 1-.06-.63l.06-.07-.06.07Z"
fill="currentColor"></path>
</svg></span></span></span></form>Text Content
We use optional cookies to improve your experience on our websites, such as
through social media connections, and to display personalized advertising based
on your online activity. If you reject optional cookies, only cookies necessary
to provide you the services will be used. You may change your selection by
clicking “Manage Cookies” at the bottom of the page. Privacy Statement
Third-Party Cookies
Accept Reject Manage cookies
Skip to content
Tech CommunityCommunity Hubs
Products
Topics
BlogsEvents
Microsoft Learn
Lounge
RegisterSign In
1. Microsoft Community Hub
2.
3. CommunitiesTopics
4.
5. Artificial Intelligence and Machine Learning
6.
7. AI - Azure AI services Blog
Report
AI - AZURE AI SERVICES BLOG
BLOG POST
AI - Azure AI services Blog
8 MIN READ
TRANSFORMING VIDEO INTO VALUE WITH AZURE AI CONTENT UNDERSTANDING
jfilcik
Microsoft
Nov 19, 2024
AZURE AI CONTENT UNDERSTANDING NOW PROVIDES ADVANCED VIDEO CAPABILITIES,
TRANSFORMING UNSTRUCTURED VIDEO INTO STRUCTURED, SEARCHABLE KNOWLEDGE. THIS
EMPOWERS BUSINESSES TO AUTOMATE VIDEO PROCESSING TASKS, EXTRACT VALUABLE
INSIGHTS, AND MAXIMIZE THE RETURN ON VIDEO INVESTMENTS, WITH LOWER DEVELOPER
OVERHEAD AND NO NEED FOR EXTENSIVE VIDEO PROCESSING CODE.
UNLOCKING VALUE FROM UNSTRUCTURED VIDEO
Every minute, social video sharing platforms see over 500 hours of video uploads
[1] and 91% of businesses leverage video as a key tool[2]. From media
conglomerates managing extensive archives to enterprises producing training and
marketing materials, organizations are overwhelmed with video. Yet, despite this
abundance, video remains inherently unstructured and difficult to utilize
effectively.
While the volume of video content continues to grow exponentially, its true
value often remains untapped due to the friction involved in making video
useful. Organizations grapple with several pain points:
* Inaccessibility of Valuable Content Archives: Massive video archives sit idle
because finding the right content to reuse requires extensive manual effort.
* The Impossibility of Personalization Without Metadata: Personalization holds
the key to unlocking new revenue streams and increasing engagement. However,
without reliable and detailed metadata, it's cost-prohibitive to tailor
content to specific audiences or individuals.
* Missed Monetization Opportunities: For media companies, untapped archives
mean missed chances to monetize content through new formats or platforms.
* Operational Bottlenecks: Enterprises struggle with slow turnaround times for
training materials, compliance checks, and marketing campaigns due to
inefficient video workflows, leading to delays and increased expenses.
Many video processing application rely on purpose-built, frame-by-frame analysis
to identify objects and key elements within video content. While this method can
detect a specific list of objects, it is inherently lossy, struggling to capture
actions, events, or uncommon objects. It also is expensive and time consuming to
customize for specific tasks.
Generative AI promises to revolutionize video content analysis, with GPT-4o
topping leaderboards for video understanding tasks, but finding a generative
model that processes video is just the first step. Creating video pipelines with
generative models is hard. Developers must invest significant effort in
infrastructure to create custom video processing pipelines to get good results.
These systems need optimized prompts, integrated transcription, smart handling
of context-window limitations, shot aligned segmentation, and much more. This
makes them expensive to optimize and hard to maintain over time.
INTRODUCING AZURE AI CONTENT UNDERSTANDING FOR VIDEO
This is where Azure AI Content Understanding transforms the game. By offering an
integrated video pipeline that leverages advanced foundational models, you can
effortlessly extract insights from both the audio and visual elements of your
videos. This service transforms unstructured video into structured, searchable
knowledge, enabling powerful use cases like media asset management and highlight
reel generation.
Content Understanding extracts specific fields from a video identifying the
location, backgrounds, and more in each segment
With Content Understanding, you can automatically identify key moments in a
video to extract highlights and summarize the full context. For example, for
corporate events and conferences you can quickly produce same-day highlight
reels. This capability not only reduces the time and cost associated with manual
editing but also empowers organizations to deliver timely, professional reaction
videos that keep audiences engaged and informed.
In another case, A news broadcaster can create a new personalized viewing
experience for news by recommending stories of interest. This is achieved by
automatically tagging segments with relevant metadata like topic and location,
enabling the delivery of content personalized to individual interests, driving
higher engagement and viewer satisfaction.
By generating specific metadata on a segment-by-segment basis, including
chapters, scenes, and shots, Content Understanding provides a detailed outline
of what's contained in the video, facilitating these workflows.
Diagram showing the Content Understanding video pipeline where video is
processed by Content Extraction and Field Extraction to create Structured
Insights
This is enabled by a streamlined pipeline for video that starts with content
extraction tasks like transcription, shot detection, key frame extraction, and
face grouping to create grounding data for analysis. Then, generative models use
that information to extract the specific fields you request for each segment of
the video. This generative field extraction capability enables customers to:
* Customize Metadata: Tailor the extracted information to focus on elements
important to your use case, such as key events, actions, or dialogues.
* Create Detailed Outlines: Understand the structure of your video content at a
granular level.
* Automate Repetitive Editing Tasks: Quickly pinpoint important segments to
create summaries, trailers, or compilations that capture the essence of the
full video.
By leveraging these capabilities, organizations can automate many video creation
tasks including creating highlight reels and repurposing content across formats,
saving time and resources while delivering compelling content to their
audiences. Whether it's summarizing conference keynotes, capturing the essence
of corporate events, or showcasing the most exciting moments in sports, Azure AI
Content Understanding makes video workflows efficient and scalable. But how do
these solutions perform in real-world scenarios?
CUSTOMER SUCCESS STORIES
IPV CURATOR: TRANSFORMING MEDIA ASSET MANAGEMENT
IPV Curator, a leader in media asset management solutions, assists clients in
managing and monetizing extensive video libraries across various industries,
including broadcast, sports, and global enterprises. It enables seamless,
zero-download editing of video in Azure cloud using Adobe applications. Their
customers needed an efficient way to search, repurpose, and produce vast amounts
of video content with data extraction tailored to specific use cases.
IPV integrated Azure AI Content Understanding into their Curator media asset
management platform. They found that it provided a step-function improvement in
metadata extraction for their clients. It was particularly beneficial as it
enabled:
* Industry Specific Metadata: Allowed clients to extract metadata tailored to
their specific needs by using simple prompts and without the need for
domain-specific training of new AI models. For example:
* Broadcast: Rapidly identified key scenes for promo production and to
efficiently identify their highest value content for Free ad-supported
streaming TV (FAST) channels.
* Travel Marketing Content: Automatically tagged geographic locations,
landmarks, shot types (e.g., aerial, close-up), and highlighted scenic
details.
* Shopping Channel Content: Detected specific products, identified demo
segments, product categories, and key selling points.
* Advanced Action and Event Analysis: Enabled detailed analysis of a set of
frames in a video segment to identify actions and events. This provides a new
level of insights compared to frame-by-frame analysis of objects.
* Segmentation Aligned to Shots: Detected shot boundaries in produced videos
and in-media edit points, enabling easy reuse by capturing full shots in
segments.
As a result, IPV's clients can quickly find and repurpose content, significantly
reducing editing time and accelerating video production at scale.
IPV Curator enables search across industry specific metadata extracted from
videos
"IPV's collaboration with Microsoft transforms media stored in Azure into an
easily accessible, streaming, and highly searchable active archive. The powerful
search engine within IPV's new generation of Media Asset Management uses Azure
AI Content Understanding to accurately surface any archived video clip, driving
users to their highest value content in seconds."
—Daniel Mathew, Chief Revenue Officer, IPV
COGNIZANT: INNOVATIVE AD MODERATION
Cognizant, a global leader in consulting and professional services, has
identified a challenge of moderating advertising content for its media
customers. Their customers' traditional methods are heavily reliant on manual
review and struggling to scale with the increasing volume of content requiring
assessment.
The Cognizant Ad Moderation solution framework leverages Content Understanding
to create a more accurate, cost-effective approach to ad moderation that results
in a 96% reduction in review time. It allows customers to automate ad reviews to
ensure cultural sensitivity, regulatory compliance, and optimizing programming
placement, ultimately reducing manual review efforts.
Ad Moderation extracts key metadata from Pepsi marketing content including
brands, content summary, brand impact, and more.
Cognizant achieves these results by leveraging Content Understanding for
multimodal field extraction, tailored output, and native generative AI video
processing.
* Multimodal Field Extraction: Extracts key attributes from both the audio and
visual elements, allowing for a more comprehensive analysis of the content.
This analysis is critical to get a holistic view of suitability for various
audiences.
* Tailored Output Schema: Outputs a custom structured schema that detects
content directly relevant to the moderation task. This includes detecting
specific risky attributes like prohibited language, potentially banned
topics, violations of content restrictions, and sensitive products like
alcohol or smoking.
* Native Generative AI Video Processing: Content Understanding natively
processes video files with generative AI to provide the detailed insights
requested in the schema capturing context, actions, and events over entire
segments of the video.
This optimized video pipeline provides Cognizant with a detailed analysis of
videos to ground an automated decision. It allows them to quickly green light
compliant ads and flag others for rejection or human review.
Content Understanding empowers Cognizant to focus on solving business challenges
rather than managing the underlying infrastructure for video processing and
integrating generative models.
“I'm absolutely thrilled about the Azure AI Content Understanding service! It's
a game-changer that accelerates processing by integrating multiple AI
capabilities into a single service call, delivering combined audio and video
transcription in one JSON output with incredibly detailed results. The ability
to add custom fields that integrate with an LLM provides even more detailed,
meaningful, and flexible output.” - Rushil Patel – Developer @ Cognizant
THE BROADER IMPACT: TRANSFORMATION ACROSS INDUSTRIES
The transformative power of Azure AI Content Understanding extends far beyond
these specific use cases, offering significant benefits across various
industries and workflows. By leveraging advanced AI capabilities on video,
organizations have been able to unlock new opportunities and drive innovation in
several key areas:
* Social Media Listening and Consumer Insights: Analyze video content across
social platforms to understand how products are perceived and discussed
online. Gain valuable consumer insights to inform product development,
marketing strategies, and brand management.
* Unlocking Video for AI Assistants and Agents: Enable AI assistants and agents
to access and utilize information from video content, transforming meeting
recordings, training videos, and events into valuable data sources for
Retrieval-Augmented Generation (RAG). Enhance customer support and knowledge
management by integrating video insights into AI-driven interactions.
* Enhancing Accessibility with Audio Descriptions: Generate draft audio
descriptions for video content to provide a starting point for human editors.
This streamlines the creation of accessible content for visually impaired
audiences, reducing effort and accelerating compliance with accessibility
standards.
* Marketing and Advertising Workflows: Automate content analysis to ensure
brand alignment and effective advertising. Understand and optimize the
content within video advertisements to maintain consistent branding and
enhance audience engagement.
The business value of Azure AI Content Understanding is clear. By addressing
core challenges in video content management with generative AI, customization,
and native video processing, it enhances operational efficiencies and unlocks
new opportunities for monetization and innovation. Organizations can now turn
dormant video archives into valuable assets, deliver personalized content to
engage audiences effectively, and automate manual time-consuming workflows.
READY TO TRANSFORM YOUR VIDEO CONTENT?
* For more details on how to use Content Understanding for video check out
the Video Solution Overview.
* If you are at Microsoft Ignite 2024 or are watching online, check out
this breakout session.
* Try this new service in Azure AI Foundry.
* For documentation, please refer to the Content Understanding Overview
For a broader perspective, see Announcing Azure AI Content Understanding:
Transforming Multimodal Data into Insights and discover how it extends these
capabilities across all content formats.
-----
[1] According to Statistia in 2022 - Hours of video uploaded every minute 2022 |
Statista
[2] According to a Wyzowl survey in 2024 - Video Marketing 2024 (10 Years of
Data) | Wyzowl
Updated Nov 19, 2024
Version 3.0
azure ai document intelligence
azure ai search
azure ai services
azure ai studio
Azure AI Video Indexer
azure ai vision
azure openai service
microsoft ignite 2024
LikeLike
0
CommentComment
jfilcik
Microsoft
Joined February 25, 2021
Send Message
View Profile
AI - Azure AI services Blog
Follow this blog board to get notified when there's new activity
SHARE
*
*
*
*
*
*
What's new
* Surface Pro 9
* Surface Laptop 5
* Surface Studio 2+
* Surface Laptop Go 2
* Surface Laptop Studio
* Surface Duo 2
* Microsoft 365
* Windows 11 apps
Microsoft Store
* Account profile
* Download Center
* Microsoft Store support
* Returns
* Order tracking
* Virtual workshops and training
* Microsoft Store Promise
* Flexible Payments
Education
* Microsoft in education
* Devices for education
* Microsoft Teams for Education
* Microsoft 365 Education
* Education consultation appointment
* Educator training and development
* Deals for students and parents
* Azure for students
Business
* Microsoft Cloud
* Microsoft Security
* Dynamics 365
* Microsoft 365
* Microsoft Power Platform
* Microsoft Teams
* Microsoft Industry
* Small Business
Developer & IT
* Azure
* Developer Center
* Documentation
* Microsoft Learn
* Microsoft Tech Community
* Azure Marketplace
* AppSource
* Visual Studio
Company
* Careers
* About Microsoft
* Company news
* Privacy at Microsoft
* Investors
* Diversity and inclusion
* Accessibility
* Sustainability
California Consumer Privacy Act (CCPA) Opt-Out Icon Your Privacy Choices
* Sitemap
* Contact Microsoft
* Privacy
* Manage cookies
* Terms of use
* Trademarks
* Safety & eco
* About our ads
* © Microsoft 2024