Provides possibility to impose different penalties on different parameters.

## Usage

```
mixedPenalty(
lavaanModel,
modifyModel = lessSEM::modifyModel(),
method = "glmnet",
control = lessSEM::controlGlmnet()
)
```

## Arguments

- lavaanModel
model of class lavaan

- modifyModel
used to modify the lavaanModel. See ?modifyModel.

- method
which optimizer should be used? Currently supported are "glmnet" and "ista".

- control
used to control the optimizer. This element is generated with the controlIsta and controlGlmnet functions. See ?controlIsta and ?controlGlmnet for more details.

## Details

The `mixedPenalty`

function allows you to add multiple penalties to a single model.
For instance, you may want to regularize both loadings and regressions in a SEM.
In this case, using the same penalty (e.g., lasso) for both types of penalties may
actually not be what you want to use because the penalty function is sensitive to
the scales of the parameters. Instead, you may want to use two separate lasso
penalties for loadings and regressions. Similarly, separate penalties for
different parameters have, for instance, been proposed in multi-group models
(Geminiani et al., 2021).

Identical to regsem, models are specified using lavaan. Currently,
most standard SEM are supported. lessSEM also provides full information
maximum likelihood for missing data. To use this functionality,
fit your lavaan model with the argument `sem(..., missing = 'ml')`

.
lessSEM will then automatically switch to full information maximum likelihood
as well. Models are fitted with the glmnet or ista optimizer. Note that the
optimizers differ in which penalties they support. The following table provides
an overview:

Penalty | Function | glmnet | ista |

lasso | addLasso | x | x |

elastic net | addElasticNet | x* | - |

cappedL1 | addCappedL1 | x | x |

lsp | addLsp | x | x |

scad | addScad | x | x |

mcp | addMcp | x | x |

By default, glmnet will be used. Note that the elastic net penalty can only be combined with other elastic net penalties.

Check vignette(topic = "Mixed-Penalties", package = "lessSEM") for more details.

Regularized SEM

Huang, P.-H., Chen, H., & Weng, L.-J. (2017). A Penalized Likelihood Method for Structural Equation Modeling. Psychometrika, 82(2), 329–354. https://doi.org/10.1007/s11336-017-9566-9

Jacobucci, R., Grimm, K. J., & McArdle, J. J. (2016). Regularized Structural Equation Modeling. Structural Equation Modeling: A Multidisciplinary Journal, 23(4), 555–566. https://doi.org/10.1080/10705511.2016.1154793

For more details on ISTA, see:

Beck, A., & Teboulle, M. (2009). A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. SIAM Journal on Imaging Sciences, 2(1), 183–202. https://doi.org/10.1137/080716542

Geminiani, E., Marra, G., & Moustaki, I. (2021). Single- and multiple-group penalized factor analysis: A trust-region algorithm approach with integrated automatic multiple tuning parameter selection. Psychometrika, 86(1), 65–95. https://doi.org/10.1007/s11336-021-09751-8

Gong, P., Zhang, C., Lu, Z., Huang, J., & Ye, J. (2013). A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems. Proceedings of the 30th International Conference on Machine Learning, 28(2)(2), 37–45.

Parikh, N., & Boyd, S. (2013). Proximal Algorithms. Foundations and Trends in Optimization, 1(3), 123–231.

## Examples

```
library(lessSEM)
# Identical to regsem, lessSEM builds on the lavaan
# package for model specification. The first step
# therefore is to implement the model in lavaan.
dataset <- simulateExampleData()
lavaanSyntax <- "
f =~ l1*y1 + l2*y2 + l3*y3 + l4*y4 + l5*y5 +
l6*y6 + l7*y7 + l8*y8 + l9*y9 + l10*y10 +
l11*y11 + l12*y12 + l13*y13 + l14*y14 + l15*y15
f ~~ 1*f
"
lavaanModel <- lavaan::sem(lavaanSyntax,
data = dataset,
meanstructure = TRUE,
std.lv = TRUE)
# Regularization:
# In this example, we want to regularize the loadings l6-l10
# independently of the loadings l11-15. This could, for instance,
# reflect that the items y6-y10 and y11-y15 may belong to different
# subscales.
regularized <- lavaanModel |>
# create template for regularized model with mixed penalty:
mixedPenalty() |>
# add lasso penalty on loadings l6 - l10:
addLasso(regularized = paste0("l", 6:10),
lambdas = seq(0,1,length.out = 4)) |>
# add scad penalty on loadings l11 - l15:
addScad(regularized = paste0("l", 11:15),
lambdas = seq(0,1,length.out = 3),
thetas = 3.1) |>
# fit the model:
fit()
# elements of regularized can be accessed with the @ operator:
regularized@parameters[1,]
# AIC and BIC:
AIC(regularized)
BIC(regularized)
# The best parameters can also be extracted with:
coef(regularized, criterion = "AIC")
coef(regularized, criterion = "BIC")
# The tuningParameterConfiguration corresponds to the rows
# in the lambda, theta, and alpha matrices in regularized@tuningParamterConfigurations.
# Configuration 3, for example, is given by
regularized@tuningParameterConfigurations$lambda[3,]
regularized@tuningParameterConfigurations$theta[3,]
regularized@tuningParameterConfigurations$alpha[3,]
# Note that lambda, theta, and alpha may correspond to tuning parameters
# of different penalties for different parameters (e.g., lambda for l6 is the lambda
# of the lasso penalty, while lambda for l12 is the lambda of the scad penalty).
```