转载自：AI科技评论 作者 | 王语霖
本文主要介绍我们被ICCV-2021会议录用为Oral Presentation的一篇文章：Adaptive Focus for Efficient Video Recognition。代码和预训练模型已经在Github上面放出，欢迎大家试用和吐槽。
太长不看版：现有高效视频识别算法往往关注于降低视频的时间冗余性（即将计算集中于视频的部分关键帧），如图1 (b)。本文则发现，降低视频的空间冗余性（即寻找和重点处理视频帧中最关键的图像区域），如图1 (c)，同样是一种效果显著、值得探索的方法；且后者与前者有效互补（即完全可以同时建模时空冗余性，例如关注于关键帧中的关键区域），如图1 (d)。在方法上，本文提出了一个通用于大多数网络的AdaFocus框架，在同等精度的条件下，相较AR-Net (ECCV-2020)将计算开销降低了2.1-3.2倍，将TSM的GPU实测推理速度加快了1.4倍。
-1-
Introduction （研究动机及简介）
相较于图像，视频识别是一个分布范围更广、应用场景更多的任务。如下图所示，每分钟，即有超过300小时的视频上传至YouTube；至2022年，超过82%的消费互联网流量将由在线视频组成。自动识别这些海量视频中的人类行为、事件、紧急情况等内容，对于视频推荐、监控等受众广泛的实际应用具有重要意义。
近年来，已有很多基于深度学习的视频识别算法取得了较佳的性能，如TSM、SlowFast、I3D等。然而，一个严重的问题是，相较于图像，使用深度神经网络处理视频通常会引入很大的计算开销。如下图所示，将ResNet-50应用于视频识别将使运算量（FLOPs）扩大8-75倍。
因此，一个关键问题在于，如何降低视频识别模型的计算开销。一个非常自然的想法是从视频的时间维度入手：一方面，相邻的视频帧之间往往具有较大的相似性，逐帧处理将引入冗余计算；另一方面，并非全部视频帧的内容都与识别任务相关。现有工作大多从这一时间冗余性出发，动态寻找视频中的若干关键帧进行重点处理，以降低计算成本，如下图所示。
但是，值得注意的一点是，我们发现，目前尚未有工作关注于视频中的空间冗余性。具体而言，在每一帧视频中，事实上只有一部分空间区域与识别任务相关，例如下图中的运动员、起跳动作、水花等。
出于这一点，本文以回答图6中的两个问题作为主线：
- 空间冗余性是否可以用于实现高效视频识别？假如我们能找到每一视频帧中的关键区域，并将主要的计算集中于这些更有价值的部分，而尽可能略过其他任务相关信息较少的区域，理论上，我们就可以显著降低网络的计算开销（事实上，我们之前基于单张图像验证过类似做法的效果：NeurIPS 2020 | Glance and Focus: 通用、高效的神经网络自适应推理框架：https://zhuanlan.zhihu.com/p/266306870）。
- 空间、时间冗余性是否互补？若上述假设成立的话，它应当可与现存的、基于时间冗余性的工作相结合，因为我们完全可以先找到少数关键帧，再仅在这些帧中寻找关键的图像区域进行重点处理。
-2-
Method （方法详述）
首先为了回答问题1，我们设计了一个AdaFocus框架，其结构如下图所示
此处我们假设视频帧按时间次序逐个输入网络，AdaFocus使用四个组件对其进行处理，
AdaFocus的训练分为三个阶段。在第一阶段，我们移除策略网络Π，随机选取局部patch，训练网络的分类部分最小化交叉熵损失，如下图所示。这一阶段意在训练网络从随机的patch序列中自适应提取信息，我们进而便可以根据分类部分网络的行为判断不同patch的重要性。
在第二阶段，我们引入随机初始化的策略网络 Π，将之建模为智能体（agent），固定分类部分，以强化学习算法对Π进行训练，优化目标为最大化折扣奖励函数。
关于强化学习的具体细节，请参见我们的paper。粗略而言，我们将选择patch建模为一个在若干个候选patch上的多类离散分布，每一类别的概率由策略网络 Π 的输出决定。训练时，我们随机从这个分布中采样出一个动作，并得到一个对应的奖励函数值（reward），reward较大的动作将受到鼓励，reward较低的动作则将受到抑制。我们将reward的值定义为给定动作在标签类别上的confidence上升的效果与随机选择patch所得效果的比较。
在第三个阶段，我们固定阶段二学到的patch选择策略，对AdaFocus的分类部分进行微调。这一阶段并非必须，但可以些微提升网络的分类性能。
到目前为止，AdaFocus对每一帧视频花费的计算量是均等的，换言之，其只建模了空间冗余性。为了探索空间、时间冗余性是否互补的问题，我们提出了增强版本的AdaFocus，AdaFocus+。具体而言，我们加入了一个与 Π结构完全相同的策略网络 Π‘ ，用于判断对于每一帧视频是否激活 fl 处理由 Π选择出的局部patch，如下图所示。
可以在训练阶段二中， Π 的训练完成后采用与其相同的强化学习算法进行训练。Π 的输出决定了一个二分类分布（Bernoulli distribution）的参数，训练时，我们随机从该分布中采样，并在采样得激活 fl 时将该操作所带来的标签类别上的confidence上升的效果与激活 fl的计算开销进行比较。只有当这一比较结果为正时，激活 fl 这一动作才会得到鼓励。
关于AdaFocus和AdaFocus+的更多细节，由于比较繁杂，不在这里赘述，可以参阅我们的paper~
-3-
Experiments （实验结果）
- 与其他高效视频识别框架的比较（基于ActivityNet）
- 在FCVID和Mini-Kinetics上的结果
- 基于TSM部署AdaFocus的结果，包含理论计算开销、CPU和GPU的实际测速结果
- AdaFocus的可视化结果，绿色方框代表我们的方法在每一帧选择的patch，可见AdaFocus成功定位到视频帧中的任务相关区域，例如帆船、自行车、做三明治的手部动作、摩托、长笛、运动员等。
- 关于网络结构设计的细节、奖励函数的设计、习得patch选择策略的有效性，我们还提供了详尽的消融实验（Ablation Study）结果，细节请参阅我们的paper。
-4-
Conclusion（结语）
总结来说，这项工作的贡献点在于：
（1）在现有的基于时间冗余性的方法之外，思考利用空间冗余性实现高效视频识别；
（2）基于强化学习，提出了一种在理论上和实测速度上效果都比较明显的通用框架，AdaFocus；
（3）在五个数据集上进行了实验，包括与其他通用框架的比较和部署于现有高效识别网络（例如TSM）上的效果等。
欢迎大家follow我们的工作~。
@InProceedings{Wang_2021_ICCV,
author = {Wang, Yulin and Chen, Zhaoxi and Jiang, Haojun and Song, Shiji and Han, Yizeng and Huang, Gao},
title = {Adaptive Focus for Efficient Video Recognition},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
@InProceedings{Wang_2021_ICCV,
author = {Wang, Yulin and Chen, Zhaoxi and Jiang, Haojun and Song, Shiji and Han, Yizeng and Huang, Gao},
title = {Adaptive Focus for Efficient Video Recognition},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021}
}
Well, I don’t know if that’s going to work for me, but definitely worked for you! 🙂 Excellent post!
Покупка диплома о среднем полном образовании: как избежать мошенничества?
A Bitcoin wallet that allows you to store, send Bitcoin, receive Bitcoin and buy Bitcoin with focus on security and simplicity Download BlueWallet for Android
Dynamic platform that aggregates live adult entertainment, constantly refreshed with streams from the leading adult webcam platforms like live gay porn
Как приобрести диплом техникума с минимальными рисками
repetitor.ekafe.ru/viewtopic.php?f=7&t=2020
I got this website from my friend who informed me concerning this website
and at the moment this time I am browsing this web
site and reading very informative articles
here.
This is a great blog. Thank you for the very informative post.
vavada bg
Сервисный центр предлагает ремонт panasonic vdr-m50 ремонт panasonic vdr-m50 в петербурге
Hey there, I think your website might be having browser compatibility
issues. When I look at your blog in Firefox, it looks fine but
when opening in Internet Explorer, it has some overlapping.
I just wanted to give you a quick heads up! Other then that, very good blog!
It is perfect time to make some plans for the future
and it’s time to be happy. I’ve read this post and if I could I desire to suggest
you few interesting things or advice. Perhaps you can write next articles
referring to this article. I desire to read more things about it!
Watch and admire beautiful girls getting naked on live cam. The site features cam girls of all types, from petite blonde camgirls to leggy redheads nudelivecam
Тут можно преобрести оружейный сейф купить сейф для охотничьего ружья
Find the hottest sexwebcam videos and clips, realy hight quality gay porno
Реально ли приобрести диплом стоматолога? Основные этапы
jivonews.ru/ne-upustite-shans-poluchite-diplom-za-minimalnyie-sroki
Приобретение диплома ПТУ с сокращенной программой обучения в Москве
bcgame
Покупка диплома о среднем полном образовании: как избежать мошенничества?
[url=https://samoylovaoxana.ru/kakoi-klimat-letom-v-tyrcii-i-chem-mojno-zaniatsia/]Какой климат летом в Турции и чем можно заняться[/url] или [url=https://samoylovaoxana.ru/s-21-iulia-karty-unionpay-ot-banka-tinkoff-prekratili-rabotat-za-rybejom/]С 21 июля карты UnionPay от банка Тинькофф прекратили работать за рубежом[/url]
[url=https://samoylovaoxana.ru/tag/kak-dobratsya-do-ligurii/]как добраться до Лигурии[/url]
https://samoylovaoxana.ru/angliiskii-dlia-detei-za-rybejom-kyda-poehat/
Ещё можно узнать: [url=http://yourdesires.ru/it/1248-kak-vvesti-znak-evro-s-klaviatury.html]евро аббревиатура[/url]
Можно ли купить аттестат о среднем образовании? Основные рекомендации
Официальное получение диплома техникума с упрощенным обучением в Москве
Base is for everyone.
Base Bridge is a cutting-edge solution facilitating smooth transfers between different blockchain networks. By creating a seamless link, it reduces the complexity and cost of blockchain transactions.
base bridge eth
What is Base Bridge?
Base Bridge is a cutting-edge solution facilitating smooth transfers between different blockchain networks. By creating a seamless link, it reduces the complexity and cost of blockchain transactions.
Key Features of Base Bridge
Interoperability: Connects multiple blockchain networks, allowing for smooth asset exchange.
Cost Efficiency: Reduces transaction fees significantly.
Security: Leverages advanced encryption to protect transactions.
Scalability: Easily adapts to increasing volumes of transactions without compromising speed.
Benefits of Using Base Bridge
Incorporating Base Bridge into your blockchain strategy offers numerous benefits:
Simplified Transactions: Eliminates the need for complicated cross-chain processes.
Faster Settlements: Enjoy quicker transaction confirmations.
Cost-Effective Operations: Lower fees mean more sustainable transactions.
Improved Security: Benefit from enhanced protection against fraud and hacking.
Аттестат школы купить официально с упрощенным обучением в Москве
Optimize your trades across hundreds of DEXes on multiple networks
1inch dao
1inch Exchange is a decentralized exchange aggregator. It searches multiple DEXs to find the most efficient path for your trade, thus minimizing costs and maximizing returns. By splitting your transaction into parts and executing them across different platforms, 1inch achieves the best possible market rates.
Explore the 1inch Network: The Future of DeFi
The 1inch Network stands at the forefront of decentralized finance (DeFi), offering traders highly efficient cryptocurrency swap solutions across multiple platforms. This introduction to 1inch will guide you through its features, benefits, and the impact it’s making in the crypto trading ecosystem.
What Makes 1inch Unique?
1inch is renowned for being a decentralized exchange aggregator. It sources liquidity from various exchanges to offer users the best possible rates. By splitting orders across multiple platforms, 1inch minimizes slippage and maximizes the value from each trade.
Core Components of 1inch
1inch Aggregation Protocol: A sophisticated routing mechanism that finds the most efficient swapping paths across multiple decentralized exchanges.
1inch Liquidity Protocol: An automated market maker (AMM) that operates in a gas-efficient way, allowing users to provide liquidity and earn a portion of trades.
1inch DAO: A decentralized autonomous organization that allows token holders to participate in the governance of the network.
Benefits of Utilizing 1inch
By tapping into 1inch’s ecosystem, traders gain access to:
Optimal Rates: The cutting-edge algorithm ensures you’re always getting the best prices available.
Security and Trust: As a non-custodial service, your funds remain under your control at all times.
Reduced Slippage: Smart order routing minimizes discrepancies in pricing.
Community-Driven: Participate in the governance processes thanks to the 1inch DAO.
How to Securely Log Into Debank: A Step-by-Step Guide
Logging into your Debank account is the first step towards managing your DeFi portfolio. Ensuring that your login credentials remain secure is crucial for protecting your investments. In this guide, we’ll walk you through the process of logging into Debank safely and efficiently.
debank check wallet
Step 1: Visit the Official Debank Website
Start by navigating to the official Debank website. Avoid using search engine results to prevent phishing attacks. The URL should be .
Step 2: Click on the Login Button
On the homepage, locate the Login button usually positioned at the top right corner of the page. Click on it to proceed to the login page.
Step 3: Enter Your Credentials
Input your registered email address and password. Ensure your password is strong, combining letters, numbers, and symbols to enhance security.
Step 4: Two-Factor Authentication
It’s recommended to enable Two-Factor Authentication (2FA) for an additional security layer. This can be done using an app like Google Authenticator. Enter the code generated by your 2FA app to continue.
Step 5: Secure Your Account
Regularly Update Your Password: Change your password every 3-6 months.
Use a Password Manager: Helps store and recall complex passwords without difficulty.
Monitor Account Activity: Regularly check for any unauthorized access or activity.
Additional Tips for Staying Safe Online
Be vigilant about where you input your credentials. Always ensure that you have a secure and reliable internet connection. Avoid accessing your Debank account on public Wi-Fi networks, which might be unsecured.
By following these guidelines, you can enhance the security of your Debank login process and safeguard your investments effectively. Stay aware of the latest security practices and updates from Debank to keep your account secure.
Explore the Aviator game for an intense betting experience where timing is key. Beginners can start with a free demo to learn the ropes, while seasoned players can enjoy the risk and potential big wins.
aviator game online aviator game real money .
Реально ли приобрести диплом стоматолога? Основные этапы
Do you have a spam issue on this site; I also am a blogger,
and I was wanting to know your situation; many of us have created some nice procedures and we are looking to exchange strategies with other folks,
why not shoot me an e-mail if interested.
Лессе-Пассе или Теудат Маавар – это oдин из возможных паспортов Израиля, который выдается гражданам этой страны https://xn—-7sbbzecaihj2bkbelhgv2p.com/
Official Starhotels website: book directly on the website at the best available rates for 4 and 5 star hotels in Italy, New York, Paris and London Grand Hotel Balkan
1win bet
Unquestionably believe that which you said. Your favorite reason seemed to be on the net the easiest thing to be aware of. I say to you, I certainly get annoyed while people consider worries that they plainly don’t know about. You managed to hit the nail on the head. Will probably be back to get more. Thanks
Тут можно преобрести оружейные сейфы москва купить оружейный сейф в интернет магазин
Как официально купить аттестат 11 класса с упрощенным обучением в Москве
Тут можно преобрести сейф противопожарный несгораемый сейф купить
This high-flying game offers a unique betting experience where Indian players can test their nerve and timing. Aviator allows for both free demo and real money play—perfect for beginners and risk-takers alike.
play aviator online aviator real money game .
Как правильно купить диплом колледжа и пту в России, подводные камни
An impressive share! I’ve just forwarded this onto a coworker
who had been doing a little research on this.
And he in fact bought me breakfast simply because I stumbled upon it for him…
lol. So let me reword this…. Thank YOU for the meal!! But
yeah, thanx for spending the time to discuss this topic here on your website.
Купить диплом Барнаул
Приобретение диплома ПТУ с сокращенной программой обучения в Москве
Всё о покупке аттестата о среднем образовании: полезные советы
Aviator combines excitement with high-stakes decisions, perfect for Indian players who love strategic gambling. Start with a demo to practice cash-out timing, then dive into real money rounds for serious rewards.
play aviator online [url=https://aviator-games-online.ru/]aviator real money game[/url] .
Одним из важнейших экологичных решений в дорожном строительстве является использование переработанных материалов. Это не только помогает сократить количество отходов, но и уменьшает потребность в новых материалах, что снижает негативное воздействие на окружающую среду Lost Cities and Forgotten Lands: Rediscovering History Through Cartography
Имплантация зубов в Вoлгoграде: 84 клиники с aдресами и ценами от 600 до 500000 рублей, запись. При выборе клиники можно ознакомиться с отзывами других безметалловые коронки на зубы Волгоград
Здесь можно преобрести сейф цена москва сейф цена
Можно ли купить аттестат о среднем образовании? Основные рекомендации
I’m amazed, I have to admit. Seldom do I come across a
blog that’s both educative and engaging, and let me tell you, you’ve hit the nail on the head.
The issue is an issue that too few folks are speaking intelligently about.
I’m very happy I stumbled across this in my hunt for something regarding this.
Как получить диплом о среднем образовании в Москве и других городах
neon54 review
Unlock the Power of Eigenlayer
As the blockchain landscape evolves, Eigenlayer emerges as a beacon of innovation, offering robust solutions to enhance blockchain security and customization. But what exactly does Eigenlayer bring to the table?
Eigenlayer
Revolutionizing Blockchain Security
Security is paramount in the blockchain world, and Eigenlayer takes this seriously. By leveraging state-of-the-art technology, Eigenlayer provides an unparalleled layer of protection for blockchain protocols. This ensures that transactions are not only secure but also reliable.
Advanced Encryption: Protects data integrity and confidentiality.
Scalable Solutions: Ensures that security measures grow with your network’s demand.
Customization like Never Before
One of the most compelling features of Eigenlayer is its customization capabilities. Understanding that each blockchain project has unique needs, Eigenlayer offers a flexible architecture allowing projects to tailor solutions to their specific requirements.
Modular Design: Adapts to various use cases without sacrificing performance.
Interoperability: Seamlessly integrates with existing blockchain systems.
Empowering Blockchain Ecosystems
By providing these advanced features, Eigenlayer doesn’t just offer protection and customization; it empowers blockchain developers to push the boundaries of what’s possible. With Eigenlayer, the limitations often faced in blockchain scalability and security are becoming a thing of the past.
Whether you are a developer seeking better security solutions or an entrepreneur aiming to launch your blockchain project with tailor-made features, Eigenlayer provides the tools and the support you need to succeed in the competitive digital landscape.
Explore the potential of Eigenlayer today, and step into the future of blockchain technology with a partner committed to your success.