| Title: | Uncertainty of Time Series Model Selection Methods |
|---|---|
| Description: | We propose a new procedure, called model uncertainty variance, which can quantify the uncertainty of model selection on Autoregressive Moving Average models. The model uncertainty variance not pay attention to the accuracy of prediction, but focus on model selection uncertainty and providing more information of the model selection results. And to estimate the model measures, we propose an simplify and faster algorithm based on bootstrap method, which is proven to be effective and feasible by Monte-Carlo simulation. At the same time, we also made some optimizations and adjustments to the Model Confidence Bounds algorithm, so that it can be applied to the time series model selection method. The consistency of the algorithm result is also verified by Monte-Carlo simulation. We propose a new procedure, called model uncertainty variance, which can quantify the uncertainty of model selection on Autoregressive Moving Average models. The model uncertainty variance focuses on model selection uncertainty and providing more information of the model selection results. To estimate the model uncertainty variance, we propose an simplified and faster algorithm based on bootstrap method, which is proven to be effective and feasible by Monte-Carlo simulation. At the same time, we also made some optimizations and adjustments to the Model Confidence Bounds algorithm, so that it can be applied to the time series model selection method. The consistency of the algorithm result is also verified by Monte-Carlo simulation. Please see Li,Y., Luo,Y., Ferrari,D., Hu,X. and Qin,Y. (2019) Model Confidence Bounds for Variable Selection. Biometrics, 75:392-403.<DOI:10.1111/biom.13024> for more information. |
| Authors: | Heming Deng Developer [aut, cre, cph], Sunan Gao Dev [aut] |
| Maintainer: | Heming Deng Developer <[email protected]> |
| License: | GPL-3 |
| Version: | 0.1.6 |
| Built: | 2025-02-26 03:30:06 UTC |
| Source: | https://github.com/cran/uotm |
The MCB algorithm can find a upper bound model (UBM) and a lower bound model (LBM) which are nested and where the true model is included between the two as a specific confidence level. And the width of the MCB is the difference between the number of variables in the UBM and LBM. The model selection method with a smaller width under the same confidence level or a larger confidence level under the same width has less uncertainty.
arma.mcb(data, max.p = 5, max.q = 5, method = "aic", stepwise = TRUE, blength = 15, btype = "mbb", bsamples = 100, seed = NA)arma.mcb(data, max.p = 5, max.q = 5, method = "aic", stepwise = TRUE, blength = 15, btype = "mbb", bsamples = 100, seed = NA)
data |
A time series sequence. |
max.p |
The max order of AR part. |
max.q |
The max order of MA part. |
method |
Information criterion to be used in model selection. |
stepwise |
If TRUE, will do stepwise selection (faster). Otherwise, it searches over all models. Non-stepwise selection can be very slow, especially for seasonal models. |
blength |
If btype is "mbb" then blength is the fixed block length used in generating the replicate time series. If btype is "sb" then blength is the mean of the geometric distribution used to generate the block lengths. blength should be a positive integer less than the length of data. |
btype |
The type of simulation required to generate the replicate time series. The possible input values are "mbb" (block resampling with fixed block lengths of blength), "sb" (block resampling with block lengths having a geometric distribution with mean blength) |
bsamples |
A positive integer giving the number of bootstrap replicates required. |
seed |
Seed. |
The arma.mcb method returns an object of class “dataframe”, which contains the bootstrap coverage rate and the corresponding model confidence bounds of the user-chosen model selection method in the form of lower confidence bound and upper confidence bound.
library(uotm) ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) tmcb <- arma.mcb(ts, max.p = 4, max.q = 4, stepwise = TRUE, bsamples = 50, blength = 12) # the model confidence bounds of time series model selection method print(tmcb)library(uotm) ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) tmcb <- arma.mcb(ts, max.p = 4, max.q = 4, stepwise = TRUE, bsamples = 50, blength = 12) # the model confidence bounds of time series model selection method print(tmcb)
The MUC of a given model selection method with good performance will tend to arch towards the upper left corner. Moreover, the area under the MUC, marked as AMUC, can be used as a raw measure of uncertainty of uncertainty for the model selection method under examination. The larger the AMUC, the stabler the corresponding model selection method. Overall, we can decide which method has the best performance according to the shape of the MUC and the AMUC.
arma.muc(method, ...)arma.muc(method, ...)
method |
Model selection method like "aic", "bic" and "aicc". |
... |
Others. |
The arma.muc method returns a figure, which is the model uncertainty curve of the model selection method.
library(uotm) ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) tmcb1 <- arma.mcb(ts, method = "aic", max.p = 4, max.q = 4, bsamples = 50, stepwise = TRUE, blength = 12) arma.muc(c("aic"), tmcb1) # compare the uncertainty of different model selection methods using muc tmcb2 <- arma.mcb(ts, method = "bic", max.p = 4, max.q = 4, bsamples = 50, stepwise = TRUE, blength = 12) arma.muc(c("aic", "bic"), tmcb1, tmcb2)library(uotm) ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) tmcb1 <- arma.mcb(ts, method = "aic", max.p = 4, max.q = 4, bsamples = 50, stepwise = TRUE, blength = 12) arma.muc(c("aic"), tmcb1) # compare the uncertainty of different model selection methods using muc tmcb2 <- arma.mcb(ts, method = "bic", max.p = 4, max.q = 4, bsamples = 50, stepwise = TRUE, blength = 12) arma.muc(c("aic", "bic"), tmcb1, tmcb2)
By calculating the MUV, we can judge the uncertainty of a model selection method and compare the uncertainty of different model selection methods. The lower the degree of uncertainty in the model selection method, the smaller its MUV.
arma.muv(data, max.p = 5, max.q = 5, method = "aic", stepwise = TRUE, blength = 15, btype = "mbb", bsamples = 100, seed = NA)arma.muv(data, max.p = 5, max.q = 5, method = "aic", stepwise = TRUE, blength = 15, btype = "mbb", bsamples = 100, seed = NA)
data |
A time series sequence. |
max.p |
The max order of AR part. |
max.q |
The max order of MA part. |
method |
Information criterion to be used in model selection. |
stepwise |
If TRUE, will do stepwise selection (faster). Otherwise, it searches over all models. Non-stepwise selection can be very slow, especially for seasonal models. |
blength |
If btype is "mbb" then blength is the fixed block length used in generating the replicate time series. If btype is "sb" then blength is the mean of the geometric distribution used to generate the block lengths. blength should be a positive integer less than the length of data. |
btype |
The type of simulation required to generate the replicate time series. The possible input values are "mbb" (block resampling with fixed block lengths of blength), "sb" (block resampling with block lengths having a geometric distribution with mean blength) |
bsamples |
A positive integer giving the number of bootstrap replicates required. |
seed |
Seed. |
The arma.muv method returns an object of class “double”, which is the model uncertainty variance of the model selection method.
library(uotm) ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) tmuv <- arma.muv(ts, max.p = 4, max.q = 4, stepwise = TRUE, bsamples = 50, blength = 12) # the model uncertainty variance of time series model selection method print(tmuv)library(uotm) ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) tmuv <- arma.muv(ts, max.p = 4, max.q = 4, stepwise = TRUE, bsamples = 50, blength = 12) # the model uncertainty variance of time series model selection method print(tmuv)
This function is to plot the time series figure.
arma.plot(data)arma.plot(data)
data |
Data containing time series. |
The arma.plot method returns a figure.
ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) arma.plot(ts)ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100) arma.plot(ts)
This function is to generate a time series sequence.
arma.sim(ars, mas, nobs, c = 0, sigma = 1, seed = NA)arma.sim(ars, mas, nobs, c = 0, sigma = 1, seed = NA)
ars |
AR part. |
mas |
MA part. |
nobs |
Length of time series sequence. |
c |
Constant. |
sigma |
The standrad error. |
seed |
Set seed. |
The arma.sim method returns an object of class “Time Series”.
ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100)ts <- arma.sim(ars = c(-0.9, -1.4, -0.7, -0.6), mas = c(0.5, -0.4), nobs = 100)