Contents. Preface. 4 Support Vector Machines Linearclassification SVMs separablecase... 64

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Contents. Preface. 4 Support Vector Machines Linearclassification SVMs separablecase... 64"

Ασπασία Θεοτόκης
6 χρόνια πριν
Προβολές:

1 Contents Preface xi 1 Introduction Applicationsandproblems Definitionsandterminology Cross-validation Learningscenarios Outline The PAC Learning Framework ThePAClearningmodel Guaranteesforfinitehypothesissets consistentcase Guarantees for finite hypothesis sets inconsistent case Generalities Deterministicversusstochasticscenarios Bayeserrorandnoise Estimationandapproximationerrors Modelselection Chapternotes Exercises Rademacher Complexity and VC-Dimension Rademachercomplexity Growthfunction VC-dimension Lowerbounds Chapternotes Exercises Support Vector Machines Linearclassification SVMs separablecase... 64

2 vi Primaloptimizationproblem Supportvectors Dualoptimizationproblem Leave-one-outanalysis SVMs non-separablecase Primaloptimizationproblem Supportvectors Dualoptimizationproblem Margintheory Chapternotes Exercises Kernel Methods Introduction Positivedefinitesymmetrickernels Definitions ReproducingkernelHilbertspace Properties Kernel-basedalgorithms SVMswithPDSkernels Representertheorem Learningguarantees Negativedefinitesymmetrickernels Sequencekernels Weightedtransducers Rationalkernels Chapternotes Exercises Boosting Introduction AdaBoost Boundontheempiricalerror Relationshipwithcoordinatedescent Relationshipwithlogisticregression Standarduseinpractice Theoreticalresults VC-dimension-basedanalysis Margin-basedanalysis Marginmaximization Game-theoreticinterpretation

3 vii 6.4 Discussion Chapternotes Exercises On-Line Learning Introduction Predictionwithexpertadvice MistakeboundsandHalvingalgorithm Weightedmajorityalgorithm Randomizedweightedmajorityalgorithm Exponentialweightedaveragealgorithm Linearclassification Perceptronalgorithm Winnowalgorithm On-linetobatchconversion Game-theoreticconnection Chapternotes Exercises Multi-Class Classification Multi-classclassificationproblem Generalizationbounds Uncombinedmulti-classalgorithms Multi-classSVMs Multi-classboostingalgorithms Decisiontrees Aggregated multi-class algorithms One-versus-all One-versus-one Error-correctioncodes Structuredpredictionalgorithms Chapternotes Exercises Ranking Theproblemofranking Generalizationbound RankingwithSVMs RankBoost Boundontheempiricalerror Relationshipwithcoordinatedescent

4 viii Margin bound for ensemble methods in ranking Bipartiteranking Boostinginbipartiteranking AreaundertheROCcurve Preference-basedsetting Second-stagerankingproblem Deterministicalgorithm Randomizedalgorithm Extensiontootherlossfunctions Discussion Chapternotes Exercises Regression Theproblemofregression Generalizationbounds Finitehypothesissets Rademachercomplexitybounds Pseudo-dimensionbounds Regressionalgorithms Linearregression Kernelridgeregression Supportvectorregression Lasso Groupnormregressionalgorithms On-lineregressionalgorithms Chapternotes Exercises Algorithmic Stability Definitions Stability-basedgeneralizationguarantee Stabilityofkernel-basedregularizationalgorithms Application to regression algorithms: SVR and KRR Applicationtoclassificationalgorithms:SVMs Discussion Chapternotes Exercises Dimensionality Reduction PrincipalComponentAnalysis

5 ix 12.2KernelPrincipalComponentAnalysis(KPCA) KPCAandmanifoldlearning Isomap Laplacianeigenmaps Locallylinearembedding(LLE) Johnson-Lindenstrausslemma Chapternotes Exercises Learning Automata and Languages Introduction Finiteautomata Efficientexactlearning Passivelearning Learningwithqueries Learningautomatawithqueries Identificationinthelimit Learningreversibleautomata Chapternotes Exercises Reinforcement Learning Learningscenario Markovdecisionprocessmodel Policy Definition Policyvalue Policyevaluation Optimalpolicy Planningalgorithms Valueiteration Policyiteration Linearprogramming Learningalgorithms Stochasticapproximation TD(0)algorithm Q-learningalgorithm SARSA TD(λ) algorithm Largestatespace Chapternotes

6 x Conclusion 339 A Linear Algebra Review 341 A.1 Vectorsandnorms A.1.1 Norms A.1.2 Dualnorms A.2 Matrices A.2.1 Matrixnorms A.2.2 Singularvaluedecomposition A.2.3 Symmetric positive semidefinite (SPSD) matrices B Convex Optimization 349 B.1 Differentiationandunconstrainedoptimization B.2 Convexity B.3 Constrainedoptimization B.4 Chapternotes C Probability Review 359 C.1 Probability C.2 Randomvariables C.3 Conditionalprobabilityandindependence C.4 Expectation, Markov s inequality, and moment-generating function. 363 C.5 VarianceandChebyshev sinequality D Concentration inequalities 369 D.1 Hoeffding sinequality D.2 McDiarmid sinequality D.3 Otherinequalities D.3.1 Binomialdistribution:Slud sinequality D.3.2 Normaldistribution:tailbound D.3.3 Khintchine-Kahaneinequality D.4 Chapternotes D.5 Exercises E Notation 379 References 381 Index 397

Σχετικά έγγραφα

Chapter 1 Introduction to Observational Studies Part 2 Cross-Sectional Selection Bias Adjustment

Contents Preface ix Part 1 Introduction Chapter 1 Introduction to Observational Studies... 3 1.1 Observational vs. Experimental Studies... 3 1.2 Issues in Observational Studies... 5 1.3 Study Design...