Refine
Document Type
- Article (3)
- In Proceedings (1)
Language
- English (4)
Has Fulltext
- no (4)
Is part of the Bibliography
- no (4)
Institute
We review various characterizations of uniform convexity and smoothness on norm balls in finite-dimensional spaces and connect results stemming from the geometry of Banach spaces with scaling inequalities used in analysing the convergence of optimization methods. In particular, we establish local versions of these conditions to provide sharper insights on a recent body of complexity results in learning theory, online learning, or offline optimization, which rely on the strong convexity of the feasible set. While they have a significant impact on complexity, these strong convexity or uniform convexity properties of feasible sets are not exploited as thoroughly as their functional counterparts, and this work is an effort to correct this imbalance. We conclude with some practical examples in optimization and machine learning where leveraging these conditions and localized assumptions lead to new complexity results.
Several learning problems involve solving min-max problems, e.g., empirical distributional robust learning
[Namkoong and Duchi, 2016, Curi et al., 2020] or learning with non-standard aggregated losses [Shalev-
Shwartz and Wexler, 2016, Fan et al., 2017]. More specifically, these problems are convex-linear problems
where the minimization is carried out over the model parameters w ∈ W and the maximization over the
empirical distribution p ∈ K of the training set indexes, where K is the simplex or a subset of it. To design
efficient methods, we let an online learning algorithm play against a (combinatorial) bandit algorithm.
We argue that the efficiency of such approaches critically depends on the structure of K and propose two
properties of K that facilitate designing efficient algorithms. We focus on a specific family of sets Sn,k
encompassing various learning applications and provide high-probability convergence guarantees to the
minimax values.
Linear bandit algorithms yield O~(n√T) pseudo-regret bounds on compact convex action sets K⊂Rn and two types of structural assumptions lead to better pseudo-regret bounds. When K is the simplex or an ℓp ball with p∈]1,2], there exist bandits algorithms with O~(√n√T) pseudo-regret bounds. Here, we derive bandit algorithms for some strongly convex sets beyond ℓp balls that enjoy pseudo-regret bounds of O~(√n√T), which answers an open question from [BCB12, §5.5.]. Interestingly, when the action set is uniformly convex but not necessarily strongly convex, we obtain pseudo-regret bounds with a dimension dependency smaller than O(√n). However, this comes at the expense of asymptotic rates in T varying between O(√T) and O(T).