Prediction of the result in race walking using ... - Semantic Scholar

Journal of Theoretical and Applied Computer Science ISSN 2299-2634 (printed), 2300-5653 (online)

Vol. 7, No. 2, 2013, pp. 45–58 http://www.jtacs.org

Prediction of the result in race walking using regularized regression models Krzysztof Przednowek1 , Krzysztof Wiktorowicz2 1 2

Faculty of Physical Education, University of Rzeszow, Poland Division of Informatics and Control, Rzeszow University of Technology, Poland

[email protected], [email protected]

Abstract:

The following paper presents the use of regularized linear models as tools to optimize training process. The models were calculated by using data collected from race-walkers’ training events. The models used predict the outcomes over a 3 km race and following a prescribed training plan. The material included a total of 122 training patterns made by 21 players. The methods of analysis include: classical model of OLS regression, ridge regression, LASSO regression and elastic net regression. In order to compare and choose the best method a cross-validation of the leave-one-out was used. All models were calculated using R language with additional packages. The best model was determined by the LASSO method which generates an error of about 26 seconds. The method has simplified the structure of the model by eliminating 5 out of 18 predictors.

Keywords:

regularized regression, shrinkage methods, variable selection, prediction of sport result, race walking, the R language

1. Introduction Today’s high performance sports are played to a very high and uniform level. Both coaches and athletes are forced to look for and use newer and sometimes innovative solutions in the process of sports training. Optimization of the process involves a number of elements, including the development of performance predictions, the selection of competitors and the creation of training plans [1, 2]. For this reason, the calculation of models which can be used to predict the results a competitor may achieve after a particular training regime is one of the methods which may optimise the training. The use of prediction models helps achieve the best selection of training components, and thus helps to achieve the desired result. Construction of regression models in the prediction of outcome is mainly used in those disciplines where contestants achieve measurable results. Many applications have been implemented in athletics events, including 400 m hurdles races as described in the work of Przednowek [3]. The calculated models performed their task of predicting outcomes for selected training phases, where three types of multiple linear regression: classic, ridge and LASSO were presented. The use of linear regression in athletics is also described in the work of Maszczyk [4], where the models predicted the outcome in javelin competition. The model was designed to act as a tool supporting the choice and selection of future javelin throwers. On the basis of a selected set of input variables the distance of a javelin throw was predicted. The presented models were classical models of multiple-regression, and Hellwig’s method was used for the selection of variables. Prediction of sport outcomes by using linear regression was also presented in the work of Przednowek and Wiktorowicz [5]. The linear prediction model incorporating ridge

Krzysztof Przednowek, Krzysztof Wiktorowicz

46

regression was used to predict the results in race walking after a direct starting preparation phase. Basic somatic features (height and weight) and training load (training components) for each day of training were built into the model from the start and the output was the expected result for a distance of 5 km. To determine the optimal parameters of the ridge regression a cross-validation method was used. A solution to a similar problem, but using distances of 5 km and 10 km was presented in the work of Przednowek [6]. However, for implementation, non-linear models were used in the form of artificial neural networks. Neural models in sports training are also used in the work of Ryguła [7, 2, 8] and Maszczyk [4, 9]. There are many applications for regression models not only in athletics, but also in other sports. Maszczyk [9] with his co-researchers used non-linear regression models to predict the results for swimmers at 50 m and 800 m. Other applications of non-linear models are described in the work of Chatterjee [10], where the regression equation was used to predict the maximal aerobic capacity of football players. As in the above-mentioned articles, different methods of constructing the regularized linear models were not considered; the objective of this work can be defined as verifying the effectiveness of these methods in predicting sports outcomes based on the example of race walking. First, the methods used to build the linear regression models, including ordinary least squares regression and regularized methods, i.e. ridge, LASSO and elastic net regressions are discussed. LASSO and elastic net regression are included in the methods which perform regularization in addition to the variable selection (shrinkage) (section 2.1-2.4). Secondly the criteria used to estimate the performance of the best model, calculated using mean square error in the process of cross-validation is defined (section 2.5). Calculating models and their evaluation in R language is also discussed (section 2.6). The article also discusses the training data of the race walkers and then regularized linear regressions will be used in order to calculate the optimal model (section 2.7). The obtained results were analysed and discussed (section 3). The success of this model will be verified by examples suggested by the expert coach (section 3.5). In order to solve the problem of the selection of the young walker’s training loads the use of regularized linear models is suggested in the article. Four methods are tested in the study: ordinary least square, ridge, LASSO and elastic net.

2. Constructing regression models We are considering the problem of constructing a multiple regression model for the set of multiple inputs Xj , j = 1, . . . , p and the one output Y shown in Figure 1. The input variables Xj are called predictors (regressors, explanatory variables), whereas the output variable Y — a response (response variable). We have assumed that it is a linear regression model in the parameters (it does not have to be linear in relation to predictors) and has the form Yˆ = w0 + X1 w1 + . . . + Xp wp p  = w0 + Xj w j ,

(1)

j=1

where Yˆ is the estimated value of the response, while w0 , wj are unknown weights (parameters) of the model. The weight w0 is called a constant term or intercept (with the y axis). We

Prediction of the result in race walking using regularized regression models

47











Figure 1. A diagram of a system with multiple inputs and one output

also assume that before making calculations the data is standardised and centred, so that the mean value of the intercept is zero and the model can be simplified to the form (see, e.g., [11]) Yˆ = X1 w1 + . . . + Xp wp p  = Xj w j .

(2)

j=1

Observations i.e. values of the variables are written as pairs (xi , yi ), i = 1, . . . , n, where xi = [xi1 , . . . , xip ]. The elements xij are the values of j-th predictor in i-th observation, where yi is the value of the response in i-th observation. For i-th observation, on the basis of formula (2), we can express it as yˆi = xi1 w1 + . . . + xip wp p  = xij wj j=1

= xi w,

(3)

where w = [w1 , . . . , wp ]T . Introducing matrix of the predictors (design matrix) described as ⎤ ⎡ x11 x12 . . . x1p ⎢ x21 x22 . . . x2p ⎥ ⎥ ⎢ X = ⎢ .. (4) .. ⎥ .. . . ⎣ . . . ⎦ . xn1 xn2 . . . xnp formula (3) can be expressed as ˆ = Xw, y

(5)

where y ˆ = [ˆ y1 , . . . , yˆn ]T . The task of constructing regression models is based on calculating the weights wj using the estimator performance criterion. In order to do this an error (residual) is introduced as the difference between the real value yi and the estimated value yˆi : ei = yi − yˆi = yi −

p 

xij wj = yi − xi w.

(6)

j=1

Using the matrix (4) we can write the error (6) as ˆ = y − Xw, e=y−y where e = [e1 , . . . , en ]T , y = [y1 , . . . , yn ]T .

(7)

Krzysztof Przednowek, Krzysztof Wiktorowicz

48

Let J(w, ·) be the criterion, which is the function of the weights and possibly other paˆ will be the vector of solutions. The problem of finding the optimal estimator rameters, then w is to minimize the criterion J(w, ·), which means solving the problem w ˆ = argmin(J(w, ·)). w

(8)

Depending on the way of defining the J(w, ·) criterion we get different models of regression. This article considers: ordinary least squares regression (OLS), ridge regression, LASSO (least absolute shrinkage and selection operator) and elastic net regression.

2.1. Ordinary least squares regression In OLS regression a popular method of least squares is used (see, e.g., [12, 11, 13]), in which weights are calculated by minimizing the sum of the squared errors J(w) =

n  i=1 n 

e2i p 

xij wj )2 .

(9)

= (y − Xw)T (y − Xw) = y − Xw22 ,

(10)

=

i=1

(yi −

j=1

In the matrix notation the criterion (9) has the form J(w) = eT e

where ·2 means the Euclidean norm (L2 ). Minimization of the criterion, which is the quadratic function of the weights w, leads to the following solution w ˆ = (XT X)−1 XT y.

(11)

It is worth mentioning that the condition for the solution (11) is the invertibility of a matrix XT X, in other words the matrix must be non-singular (non degenerate). A problem, which does not fulfil this condition is called ill-posed or ill-conditioned. There are two reasons for such a situation to occur: — a number of the searched parameters wj , or predictors, are greater than the number of observation (xi , yi ), so p > n, — the columns of the X matrix are linearly dependent (e.g. when X2 = 3X1 ), so the predictors are correlated. In order to improve conditioning, different methods of regularization of the models are used, the most popular include: Principal Component Regression (PCR), Partial Least Squares (PLS), or other types of regression with constraints including the previously-mentioned: ridge, LASSO and elastic net regressions.

Prediction of the result in race walking using regularized regression models

49

2.2. Ridge regression In ridge regression [14] the criterion of performance includes a penalty for increased weights and takes the form J(w, λ) =

n  i=1 n 

e2i + λ

p 

wj2

j=1 p



p 

wj2

(12)

= (y − Xw)T (y − Xw) + λwT w = y − Xw22 + λw22 .

(13)

=

(yi −

i=1

2

xij wj ) + λ

j=1

j=1

or in matrix notation J(w, λ) = eT e + λwT w

Parameter λ ≥ 0 decides the size of the penalty: the greater the value λ the bigger the penalty; for λ = 0 ridge regression is reduced to OLS regression. Solving the problem (8) for ridge regression takes the form w ˆ = (XT X + λI)−1 XT y,

(14)

where I is an identity matrix with the size p×p. As can be seen, the ridge regression increases the diagonal of the matrix XT X by a positive constant. Thanks to this the problem becomes non-singular i.e. the matrix XT X + λI is invertible. In addition, as in OLS regression the solution is linear in relation to y.

2.3. LASSO regression LASSO regression [15], similarly to ridge regression, adds to the criterion of performance penalty, where instead of L2 the norm L1 is used i.e. the sum of absolute values: J(w, λ) = =

n  i=1 n  i=1

e2i

+λ

p 

|wj |

j=1 p

(yi −



xij wj )2 + λ

j=1

p 

|wj |.

(15)

j=1

Using matrix notation we get J(w, λ) = eT e + λzT w = (y − Xw)T (y − Xw) + λzT w = y − Xw22 + λw1 .

(16)

where z = [z1 , . . . , zp ]T is a vector with elements zj = sign(wj ), whereas ·1 signifies the Manhattan norm (L1 ). Due to the fact that the norm L1 is used as penalty, solving the problem (8) is not linear in relation to y and it cannot be presented in the compact form as in the case of ridge regression. In order to obtain the solution various algorithms are used, and the most popular of them is the LARS algorithm (least angle regression) [16], which finds the solution (piecewise linear function) for any λ ≥ 0.

Krzysztof Przednowek, Krzysztof Wiktorowicz

50

2.4. Elastic net regression Elastic net regression (ENET) [17] combines the features of ridge and LASSO regressions. The performance criterion in the so-called naïve elastic net takes the form of J(w, λ1 , λ2 ) =

n 

e2i

+ λ1

i=1

=

n  i=1

p 

|wj | + λ2

j=1

(yi −

p  j=1

p 

wj2

j=1 p 2

xij wj ) + λ1

 j=1

|wj | + λ2

p 

wj2 .

(17)

j=1

or in matrix notation J(w, λ1 , λ2 ) = eT e + λ1 zT w + λ2 wT w = (y − Xw)T (y − Xw) + λ1 zT w + λ2 wT w = y − Xw22 + λ1 w1 + λ2 w22 .

(18)

To minimize the criterion J(w, λ1 , λ2 ) Zou and Hastie [17] suggested the LARS-EN algorithm, which is based on the LARS algorithm for LASSO regression. They made use of the fact that elastic net regression can be reduced to LASSO regression for the so-called augmented data set (X∗ , y∗ ).

2.5. Choosing the model In order to choose the best model a method of cross-validation [18] is used. In this method a data set is divided into two subsets: the training set and the testing (validation) set, the first of which is used to construct models, the second to evaluate the quality. Depending on the division of the data set a variety of methods are distinguished, the most popular are: k-fold cross-validation and leave-one-out cross-validation (LOOCV). In this paper, due to the small amount of data, LOOCV was selected, in which a testing set consists of one pair of selected data (xi , yi ), whereas the number of tests is equal to the number of data n. As the performance criterion of the model the square root of the mean square error was calculated on the basis of the formulas n 1 MSECV = (yi − yˆ−i )2 , (19) n i=1 (20) RMSECV = MSECV , where: yˆ−i — the output value of the model constructed in the i-th step of cross-validation based on a data set containing no testing pair (xi , yi ), MSECV — mean square error, RMSECV — root mean square error. In addition to the cross-validation error, which allows evaluation of the predictive ability of the model, a training error — describing the measure to which it matches the data — will be considered as well. A training error is defined as n 1 MSET = (yi − yˆi )2 , (21) n i=1 (22) RMSET = MSET , where yˆi — the output value of the model built in the i-th step, based on the full data set, MSET — mean square error of training, RMSET — root mean square error of training.

Prediction of the result in race walking using regularized regression models

51

2.6. Implementation of calculations All the regression models were calculated using R language with additional packages [19]. The lm.ridge function from ’MASS’ package [20] was used for calculating OLS regression (where λ = 0) and ridge regression (where λ > 0). With the function enet included in the package ’elastic net’ [21] LASSO regression and elastic net regression were calculated. The parameters of the enet function are: s ∈ [0, 1] and λ ≥ 0, where s is a fraction of the L1 norm, whereas λ signifies λ2 in the formulas (17) and (18). Parametrization of elastic net regression using a pair (λ, s) instead of (λ1 , λ2 ) present in the formulas (17) and (18) is possible, because the elastic net regression can be treated as the LASSO regression for an artificial data set (X∗ , y∗ ) [17]. Assuming that λ = 0 we get LASSO regression with one parameter s for the original data (X, y). Parameters in all regressions were selected on a leave-one-out cross-validation, for which its own functions were prepared. The primary performance criterion of the model was RMSECV error.

2.7. Training data The training data of athletes practising race walking was used to build the models. A group of juniors and colts from Podkarpackie province participated in the survey. Among the tested competitors were the finalists in the Polish Junior Championships and the Polish Junior Indoor Championships. The training data was recorded during the 2011-2012 season in the form of training means and the value of training loads. Training load is the amount of work at a particular intensity done by a player during exercise while the training means is the type of work performed [22]. In the material which has been gathered, and following the assumptions set out in the work of Kisiel [23], 11 means of training were distinguished. The material was drawn from the records of the annual training cycle, in which four specific phases were distinguished: transition, general preparation, special preparation, and starting periods (phase). The training data was recorded in the form of sums of training loads completed in one month of the selected training phase. The material included a total of 122 training patterns made by 21 players. Control of the training process in race walking requires various tests of physical fitness, different at every training level. In this paper, the research material are the competitors in colt and junior categories. Therefore, in order to determine a unified criterion of the level of training, a score obtained for 3000 m race walking was used. The distance of 3000 m is justified by the fact that this is the indoor walking competition [23]. Table 1 gives an accurate description of the variables under consideration and their basic statistics, i.e. the arithmetic mean of x¯, the minimum value xmin , maximum value xmax , standard deviation s and coefficient of variation V = sd/¯ x · 100%. Variables X1 , X2 , X3 and X4 are qualitative variables with values from the set {0, 1}; the other variables are quantitative variables. Training periods are represented by inputs X1 , X2 , X3 , where the value of 1, indicating the choice of a particular stage of training, may be only one of these inputs. If the only value at inputs X1 , X2 , X3 is 0, it means that the prediction applies to the transitional period. The gender of the competitor is represented by a variable X4 , where the value 0 signifies a female, and value 1 signifies a male. Basic somatic features of the athletes such as height and weight are presented in the form of BMI (body mass index) expressed by the formula BMI = M/H 2 [kg/m2 ], (23)

Krzysztof Przednowek, Krzysztof Wiktorowicz

52

Table 1. The variables and their basic statistics Variable y X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 X17 X18

Description Result over 3 km [s] General preparation phase Special preparation phase Starting phase Competitor’s gender Competitor’s age [years] BMI (body mass index) [kg/m2 ] Current result over 3 km [s] Overall running endurance [km] Overall walking endurance in 1 intensity range [km] Overall walking endurance in 2 intensity range [km] Overall walking endurance in 3 intensity range [km] Short tempo endurance [km] Medium tempo endurance [km] Long tempo endurance [km] Exercises forming technique (rhythm) of walking [km] Exercises forming muscle strength [min] Exercises forming general fitness [min] Universal exercises (warm up) [min]

x ¯ 936.9 18.9

xmin 780 14

xmax 1155 24

sd 78.4 3.0

V [%] 8.4 15.6

19.3

16.4

22.1

1.7

8.7

962.6 30.9

795 0

1210 56

87.7 10.6

9.1 34.4

224.6

57

440

96.1

42.8

53.2

0

120

34.6

65.1

7.9

0

30

9.4

119.7

8.9 8.3 12.9

0 0 0

24 32.4 56

5 8.6 16.1

56.0 103.2 125.0

4.4

0

12

4.2

96.0

90.2

0

360

104.8

116.3

522.0

120

720

109.9

21.0

317.3

150

420

72.5

22.8

where: M — body weight in kilograms, H — body height in metres. Training means (training) are represented by the variables from X8 to X18 . These include running exercises (X8 ), walking continuously with different levels of intensity (X9 , X10 , X11 ), exercises forming different types of endurance (X12 , X13 , X14 ), exercises forming techniques (X15 ), muscle strength (X16 ), general fitness (X17 ) and warming up exercises (X18 ). An example of a training model has the form: x5 = [0, 1, 0, 0, 23, 22.09, 800, 32, 400, 112, 20, 16, 32.4, 48, 8, 280, 640, 400], y5 = 800.

(24)

Vector x5 describes a 23-year-old athlete with BMI = 22.09 [kg/m2 ], who in the preparation period of special training completes training described by the variables from X8 to X18 . Both before and after the training the result was the same and amounted to 800 s.

3. Results analysis and discussion The task of predicting results from a coach’s point of view is very important in the process of sport training. A coach using the model which was constructed earlier can predict how the set training will influence the sport outcome. The presented models make predictions based

Prediction of the result in race walking using regularized regression models

53

Table 2. Coefficients of models and error results LASSO, Regression OLS RIDGE ENET w0 237.2 325.7 296.6 w1 45.67 34.67 32.75 w2 90.61 74.84 71.91 w3 39.70 27.49 24.45 w4 −2.838 2.424 w5 −0.9755 −1.770 −1.416 w6 1.072 0.5391 w7 0.7331 0.6805 0.7069 w8 −0.2779 −0.3589 −0.3410 w9 −0.1428 −0.1420 −0.1364 w10 −0.1579 −0.0948 −0.0200 w11 0.7472 0.4352 0.0618 w12 0.4845 0.3852 0.1793 w13 0.1216 0.1454 0.1183 w14 −0.1510 −0.0270 w15 −0.5125 −0.3070 w16 −0.0601 −0.0571 −0.0652 w17 −0.0153 −0.0071 w18 −0.0115 −0.0403 −0.0220 RMSECV [s] 26.90 26.76 26.20 RMSET [s] 22.70 22.82 22.89

on the proposed monthly training introduced as the sum of the training loads of each type implemented in a given month. The results of the research were presented in Table 2, the description of the selected regressions will be presented in the next paragraphs.

3.1. OLS regression The model of regression calculated by the OLS method generates a prediction error RMSET = 22.70 s (Table 2). The intercept of w0 and weights wj are presented in the second column of Table 2.

3.2. Ridge regression The search for the optimal ridge regression model is based on finding a value for λ, for which the constructed model will achieve the smallest prediction error. In the article ridge regression models for parameter λ were analysed, which was changing from 0 to 2 with steps of 0.1 (Figure 2). On the basis of the results obtained it was found that the best ridge model is obtained for λopt = 1. The prediction error was smaller than in the OLS model and amounted to RMSECV = 26.76 s, while the training error was greater and equalled RMSET = 22.82 s (Table 2). The result is that the ridge regression improved the predictive ability of the model at the expense of a worse fit with the teaching data. On the basis of the graphs presented in Figure 2 it can be stated that the fitting of the model to the data can be improved i.e. decreasing RMSET error by decreasing the penalty imposed on the model. However it would be at the cost of increasing the prediction error. Figures 3 demonstrate the dependence of the value of the intercept w0 and weights wj of the model and parameter λ. The vertical line marks

Krzysztof Przednowek, Krzysztof Wiktorowicz

54 a)

b) RMSECV [s]

RMSECV [s]

26.9 26.85 26.8 26.75 0

0.5

1 λ

1.5

27 0.6

s

0.8

1

0.8

1

28 RMSET [s]

23

T

RMSE [s]

28

26 0.4

2

23.1

22.9 22.8 0

29

0.5

1 λ

1.5

2

26 24 22 0.4

0.6

s

Figure 2. Cross-validation error (RMSECV ) and training error (RMSET ) for: a) ridge regression for parameter λ; the point marks the minimum of RMSECV = 26.76 s, where λopt = 1, b) LASSO regression for parameter s; the point marks the minimum of RMSECV = 26.2 s, where sopt = 0.78

an optimal model for λopt = 1, while the exact values of the parameter of the model are in the third column of Table 2. It is seen that as in the case of OLS regression, all weights are non-zero, thus all the input variables are involved in calculating the input of the model.

3.3. LASSO regression The LASSO regression model was calculated using the LARS-EN algorithm, in which the size of the penalty is associated with the size of parameter s increasing the value from 0 to 1 (Figure 2). After analysing models for parameter s with step changes of 0.01 it was found that the optimal LASSO regression fulfilling the task of predicting the result over the distance of 3 km is calculated for sopt = 0.78. The relationships between learning and prediction errors and the parameter s are illustrated in Figure 2. The best model of LASSO regression generates an error RMSECV = 26.20 s, which improves the result obtained by OLS and ridge models. However, it should be emphasized that this model is characterized by the worst data fit as indicated by the greatest learning error RMSET = 22.89 s. The LASSO method, beside regularizing the model, is used for calculating an optimal set of input variables (shrinkage). It can be seen, on the basis of the fourth column of Table 2, that the LASSO regression eliminated from the model the 5 input variables (X4 , X6 , X14 , X15 and X17 ), which made the model simpler than for OLS and ridge regression. Figure 4 presents profiles of the intercept w0 and weights wj of the model plotted in relation to parameter s. A vertical line indicates the optimal model for sopt = 0.78. It is also worth mentioning that the weights profiles are piecewise linear.

3.4. Elastic net regression The use of elastic net regression yielded no improvement in terms of reducing the value of the prediction error. The best regression model was achieved for a pair of parameters

Prediction of the result in race walking using regularized regression models

400

w

0

1

350

0.5

300

0

250

−0.5

200 0

0.5

1 λ

1.5

−1 0

2

w11 w10 w9 w 8

0.5

1 λ

1.5

2

0.6

100 80 60

0.4 w

w

0.2

w w13

w w1 3 w

−0.2

2

12

0

40 20

4

0 0

55

0.5

1 λ

1.5

15

−0.4 0

2

14

w

0.5

1 λ

1.5

2

0 1

w 7 w 6

0

−0.02 −0.04

−1 −2 0

w17

w 0.5

1 λ

1.5

2

5

w16 w18

−0.06 0

0.5

1 λ

1.5

2

Figure 3. Intercept w0 and weights w1 , . . . , w7 of the model for ridge regression for parameter λ; a vertical line drawn for λopt = 1, signifies the model chosen in cross-validation

sopt = 0.78 and λopt = 0 (Figure 5). Due to the fact that parameter λ is zero, the model is identical to the LASSO regression (fourth column of Table 2).

3.5. Prediction of the result The use of the model relies on the coach introducing competitor’s input parameters and the training loads. The model generates the expected result for 3000 m. If the generated result is unsatisfactory the coach can change the values of the selected training loads by introducing appropriate changes. Modified training goes back to the input, then the system generates the result obtained by a competitor for the modified training. This allows the coach to observe the changes resulting from the modification of individual components of the training which will enable the selection of the optimal training loads. The conducted analysis showed that the best model for performing the task of predicting outcomes in race walking is the LASSO regression model with parameter s = 0.78. In order to further validate the model, its operation was checked through consultation with an expert. In the model-building stage, 3 exemplary data suggested by the coach were entered (the second and third column of Table 3). Next columns are: estimated values, errors in seconds (y − yˆ) and relative errors in percentage (|y − yˆ|/sd · 100%). From the perspective of a coach such an error is acceptable and it seems that these types of models can be used in predicting sport outcomes.

Krzysztof Przednowek, Krzysztof Wiktorowicz

56

1

450

w11

400

0.5

350 300

w 10 w9 w

0

250

w

0

200 0.4

0.5

0.6

0.7 s

0.8

0.9

1

100

8

−0.5 0.4

2

60

w 1 w

40

3

20 0

w

4

−20 0.4

0.5

0.6

0.7 s

0.8

0.9

0 w5

−1 −2

0.9

1

w12 w13

0

w14

−0.2 −0.4

w15

0.4

7

0.8

0.2

w 6 w

1

0.7 s

0.4

1

2

0.6

0.6

w

80

0.5

0.5

0.6

0.7 s

0.8

0.9

1

0

w18 w17

−0.02 −0.04

w

−0.06

−3 0.4

0.5

0.6

0.7 s

0.8

0.9

16

0.4

1

0.5

0.6

0.7 s

0.8

0.9

1

Figure 4. Intercept w0 and weights w1 , . . . , w18 of the model for LASSO regression for parameter s; vertical line drawn for sopt = 0.78, signifies a model chosen in cross-validation

4. Summary This paper presents regularized linear models used to predict sports results for athletes practising race walking. By introducing a monthly training schedule for a selected phase in the annual cycle the possibility of a fall in physical form may be predicted based on the results this method has generated. This makes it possible to make earlier changes in the scheduled training. This paper examines the use of regularized models in order to reduce the prediction error generated by the classical model of OLS. The best model was determined by the LASSO method which generates an error of about 26 seconds. Taking into account the fact that the prediction error amounts to 33.42 % of the standard deviation result over a distance of 3 km, Table 3. Predicting results No. 1 2 3

Input data xT [0, 0, 0, 0, 15, 15.6, 1230, 44, 77, 10, 0, 12.8, 0, 0, 0, 0, 430, 300] [1, 0, 0, 0, 21, 21.4, 910, 24, 262, 72, 0, 4, 0, 10, 0, 160, 560, 360] [0, 1, 0, 1, 16, 18.08, 990, 52, 210, 56, 13, 6.4, 16, 24, 8, 0, 480, 320]

y [s]

yˆ [s]

Error [s]

Error [%] sd

1135

1116.25

18.75

23.92

903

879.9

23.1

29.46

982

994.99

−12.99

16.56

Prediction of the result in race walking using regularized regression models

80 RMSE [s]

RMSE

60

T

CV

[s]

80

57

40 20

60 40 20

1 0.5 0

λ

0.4

0.2

0.5

CV

[s]

35

39

s

T

47 51

55

59

0.8

43

51

31 31

0.4

27

0.6 s

0.2 1

35

31

47 43

27

0.8

39

31

0.4

35

31

35

0.6

51

0.4

39

λ

λ

35

47

35

0.6

0 0.2

51

39 47

47 43 39 35

55

43

43

0.2

59

47

35

1

35

0.8

0.4

0.2

1

0.8

0.6

RMSE [s]

39 43 47

43 47

43

39

0

λ

s

RMSE

1

0.8

0.6

1

1

0 0.2

27

27 39 35

31 27

0.4

0.6 s

0.8

1

Figure 5. Cross-validation error (RMSECV ) and training error (RMSET ) for elastic net regression for parameters s and λ; a point marks a minimum of RMSECV = 26.2 s with optimal values of sopt = 0.78, λopt = 0

it confirms the validity of the use of such methods in predicting sports results. In addition, the method has simplified the structure of the model by eliminating 5 out of 18 predictors. Experts in the fields of sports theory and training were consulted during the construction of the models in order to maintain the theoretical and practical principles of sport training. The next stage of work will be to examine the efficacy of non-linear models in predicting the outcome over a distance of 3 km in race walking. In addition, testing of the model obtained during a training course will be conducted. Making the generated model available to a coach will enable collecting comments to be taken into account while constructing the next versions of the system. The authors intend to compare the errors generated by the designated linear models with errors generated by various types of non-linear models, including non-linear regression and neural networks.

References [1] Soza´nski, H.: Directions of training loads optimization (in Polish). Akademia Wychowania Fizycznego, Warszawa, 1992. [2] Ryguła, I.: Tools of the system analysis in the sports-training (in Polish). Akademia Wychowania Fizycznego, Katowice, 2002.

58

Krzysztof Przednowek, Krzysztof Wiktorowicz

[3] Przednowek, K., Iskra, J., Cieszkowski, S.: The use of selected linear models in predicting the results of 400-metre hurdles races. Current research in motor control, 4, 2012. [4] Maszczyk, A., Zajac, ˛ A., Ryguła, I.: A neural network model approach to athlete selection. Sports Engineering, 13(1), pp. 83–93, 2011. [5] Przednowek, K., Wiktorowicz, K.: Neural system of sport result optimization of athletes doing race walking (in Polish). Metody Informatyki Stosowanej, 29(4), pp. 189–200, 2011. [6] Przednowek, K., Cieszkowski, S., Wiktorowicz, K.: Expert system in sport training (in Polish). Sport Wyczynowy, 538(2), pp. 27–32, 2011. [7] Ryguła, I.: Neural models as tool of sport prediction. Journal of Human Kinetics, 4(1), pp. 133–146, 2000. [8] Ryguła, I.: Artifical Neural Networks As a Tool of Modeling of Training Loads. Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, 1(1), pp. 2985–2988, 2005. [9] Maszczyk, A., Roczniok, R., Wa´skiewicz, Z., Czuba, M., Mikołajec, K., Zajac, ˛ A., Stanula, A.: Application of regression and neural models to predict competitive swimming performance. Peceptual and Motor Skills, 114(2), pp. 610–626, 2012. [10] Chatterjee, P., Banerjee, A. K., Das, P., Debnath, P.: A Regression Equation to Predict VO2 Max of Young Football Players of Nepal. International Journal of Applied Sports Sciences, 21(2), pp. 113–121, 2009. [11] Hastie, T., Tibhsirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, 2009. [12] Maddala, G. S.: Introduction to Econometrics. Wiley, Chichester, 2001. [13] Bishop, C. M.: Pattern Recognition and Machine Learning. Springer, New York, 2006. [14] Hoerl, A. E., Kennard, R. W.: Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12(1), pp. 55–67, 1970. [15] Tibshirani, R.: Regression Shrinkage and Selection via the Lasso. Journal of The Royal Statistical Society. Series B (Methodological), 58(1), pp. 267–288, 1996. [16] Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression (with discussion). The Annals of Statistics, 32(2), pp. 407–499, 2004. [17] Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 67(2), pp. 301–320, 2005. [18] Arlot, S., Celisse, A.: A survey of cross-validation procedures for model selection. Statistics Surveys, 4, pp. 40–79, 2010. [19] R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2011. [20] Ripley, B.: Package ’MASS’, 2012. http://cran.r-project.org/web/packages/ MASS/MASS.pdf. [21] Zou, H., Hastie, T.: Package ’elasticnet’, 2012. http://cran.r-project.org/web/ packages/elasticnet/elasticnet.pdf. [22] Soza´nski, H.: Foundations of sports training theory (in Polish). Biblioteka Trenera, Warszawa, 1999. [23] Kisiel, K.: Race Walking (in Polish). Biblioteka Trenera, Warszawa, 2008.