# Algorithms for a Partially Regularized Least Squares Problem

Statistical analysis of data from rivers deals with time series which are dependent, e.g., on climatic and seasonal factors. For example, it is a well-known fact that the load of substances in rivers can be strongly dependent on the runoff. It is of interest to find out whether observed changes in riverine loads are due only to natural variation or caused by other factors. Semi-parametric models have been proposed for estimation of time-varying linear relationships between runoff and riverine loads of substances. The aim of this work is to study some numerical methods for solving the linear least squares problem which arises.The model gives a linear system of the form A1x1 + A2x2 + n = b1. The vector n consists of identically distributed random variables all with mean zero. The unknowns, x, are split into two groups, x1 and x2. In this model, usually there are more unknowns than observations and the resulting linear system is most often consistent having an infinite number of solutions. Hence some constraint on the parameter vector x is needed. One possibility is to avoid rapid variation in, e.g., the parameters x2. This can be accomplished by regularizing using a matrix A3, which is a discretization of some norm. The problem is formulated as a partially regularized least squares problem with one or two regularization parameters. The parameter x2 has here a two-dimensional structure. By using two different regularization parameters it is possible to regularize separately in each dimension.We first study (for the case of one parameter only) the conjugate gradient method for solution of the problem. To improve rate of convergence blockpreconditioners of Schur complement type are suggested, analyzed and tested. Also a direct solution method based on QR decomposition is studied. The idea is to first perform operations independent of the values of the regularization parameters. Here we utilize the special block-structure of the problem. We further discuss the choice of regularization parameters…

Contents

1 Introduction
2 Solving the linear least squares problem
1 The least squares problem
1.1 Overdetermined systems
1.2 Underdetermined systems
2 Algorithms for computing the least squares solution
2.1 Direct methods
2.2 Iterative methods
3 The QR decomposition and least squares
3.1 Overdetermined system
3.2 Underdetermined system
4 The conjugate gradient method CG and CGLS
5 Convergence properties of CGLS
5.1 Singular values in two arbitrary intervals
6 The conjugate gradient method with preconditioning
3 A partially regularized least squares problem
1 An underdetermined system of equations
1.1 The model
1.2 Matrix formulation
1.3 Data
1.4 The minimum norm solution
2 Partially regularization
2.1 The regularized model
2.2 Examples
3 Choice of regularization parameters
4 Summary of papers
References

Author: Skoglund, Ingegerd