The Nature of Mathematical Modeling. Neil Gershenfeld

Transcript

1 The Nature of Mathematical Modeling Neil Gershenfeld

2 PUBLISHED BY THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE The Pitt Building, Trumpington Street, Cambridge CB2 1RP, United Kingdom CAMBRIDGE UNIVERSITY PRESS The Edinburgh Building, Cambridge CB2 2RU, UK 40 West 20th Street, New York, NY , USA 10 Stamford Road, Oakleigh, Melbourne 3166, Australia c Cambridge University Press 1999 This book is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 1999 Printed in the United Kingdom at the University Press, Cambridge TypesetinMonotypeEhrhardt /13pt,inL ATEX2ε[EPC] A catalogue record of this book is available from the British Library Library of Congress Cataloguing in Publication data Gershenfeld, Neil A. The nature of mathematical modeling/ Neil Gershenfeld. p. cm. Includes bibliographical references and index. ISBN Mathematical models. I. Title. QA401.G dc CIP ISBN hardback

3 Contents Preface to the second edition Preface to the first edition page xiii xv 1 Introduction SelectedReferences Part One: Analytical Models 5 2 Linear Algebra Numbers,Vectors,andMatrices SystemsofEquations SingularValues FunctionsofMatrices SelectedReferences Problems Ordinary Differential and Difference Equations LinearDifferentialEquations SystemsofDifferentialEquationsandNormalModes LaplaceTransforms PerturbationExpansions DiscreteTimeEquations z-transforms SelectedReferences Problems Partial Differential Equations TheOriginofPartialDifferentialEquations LinearPartialDifferentialEquations SeparationofVariables RectangularCoordinates CylindricalCoordinates SphericalCoordinates TransformTechniques

4 DRAFT Contents vii 4.5 SelectedReferences Problems Variational Principles VariationalCalculus Euler sequation IntegralsandMissingVariables ConstraintsandLagrangeMultipliers VariationalProblems Optics:Fermat sprinciple AnalyticalMechanics:Hamilton sprinciple Symmetry:Noether stheorem RigidBodyMotion SelectedReferences Problems Random Systems RandomVariables JointDistributions CharacteristicFunctions Entropy StochasticProcesses DistributionEvolutionEquations StochasticDifferentialEquations RandomNumberGenerators LinearCongruential LinearFeedback RandomAlgorithms SelectedReferences Problems Part Two: Numerical Models 77 7 Finite Differences: Ordinary Differential Equations NumericalApproximations Runge KuttaMethods BeyondRunge Kutta SelectedReferences Problems Finite Differences: Partial Differential Equations HyperbolicEquations:Waves ParabolicEquations:Diffusion EllipticEquations:BoundaryValues SelectedReferences

5 viii Contents DRAFT 8.5 Problems Finite Elements WeightedResiduals Rayleigh RitzVariationalMethods BoundaryConditions SolidMechanics SelectedReferences Problems Cellular Automata and Lattice Gases LatticeGasesandFluids CellularAutomataandComputing SelectedReferences Problems Computational Geometry Meshes Distances Shapes Graphs Folding ? SelectedReferences Problems Part Three: Data-Driven Models Function Fitting ModelEstimation LeastSquares LinearLeastSquares SingularValueDecomposition NonlinearLeastSquares Levenberg MarquardtMethod Errors Estimation, Fisher Information, and the Cramér Rao Inequality SelectedReferences Problems Transforms OrthogonalTransforms FourierTransforms Wavelets PrincipalComponents IndependentComponents

6 DRAFT Contents ix 13.6 SelectedReferences Problems Architectures Polynomials PadéApproximants Splines OrthogonalFunctions RadialBasisFunctions Overfitting CurseofDimensionality NeuralNetworks BackPropagation Regularization SelectedReferences Problems Optimization MultidimensionalSearch DownhillSimplex LineMinimization DirectionSet ConjugateGradient StochasticSearch LocalMinima SimulatedAnnealing GeneticAlgorithms TheBlessingofDimensionality SelectedReferences Problems Density Estimation Histogramming,Sorting,andTrees FittingDensities Mixture Density Estimation and Expectation-Maximization Cluster-WeightedModeling ProbabilisticNetworks SelectedReferences Problems Constrained Optimization LagrangeMultipliers Optimality Solvers Penalty AugmentedLagrangian

7 x Contents DRAFT InteriorPoint SelectedReferences Problems Convex Optimization Kernels SupportVectorMachines Classification Regression Clustering Relaxations SelectedReferences Problems Part Four: Dynamical Models Filtering and State Estimation MatchedFilters WienerFilters KalmanFilters NonlinearityandEntrainment HiddenMarkovModels GraphicalNetworks SelectedReferences Problems Linear and Nonlinear Time Series LinearTimeSeries TheBreakdownofLinearSystemsTheory State-SpaceReconstruction Characterization Dimensions LyapunovExponents Entropies Forecasting SelectedReferences Problems Control Theory Problems Appendix 1 Graphical and Mathematical Software 278 A1.1 MathPackages A1.1.1 ProgrammingEnvironments A1.1.2 InteractiveEnvironments A1.2 GraphicsTools

8 DRAFT Contents xi A1.2.1 Postscript A1.2.2 XWindows A1.2.3 OpenGL A1.2.4 Java A1.2.5 *ification A1.3 Problems Appendix 2 Network Programming 307 A2.1 OSI,TCP/IP,andAllThat A2.2 SocketI/O A2.3 ParallelProgramming Appendix 3 Benchmarking 314 Appendix 4 Problem Solutions 316 A4.1 Introduction A4.2 LinearAlgebra A4.3 OrdinaryDifferentialandDifferenceEquations A4.4 PartialDifferentialEquations A4.5 VariationalPrinciples A4.6 GeometryandSymmetry A4.7 RandomSystems A4.8 FiniteDifferences:OrdinaryDifferentialEquations A4.9 FiniteDifferences:PartialDifferentialEquations A4.10FiniteElements A4.11CellularAutomataandLatticeGases A4.12FunctionFitting A4.13Transforms A4.14Architectures A4.15Search A4.16DensityEstimation A4.17ConstrainedOptimization A4.18ConvexOptimization A4.19FilteringandStateEstimation A4.20LinearandNonlinearTimeSeries A4.21ControlTheory Bibliography 405 Index 417