How to obtain Confidence Intervals for a LASSO regression? The 2019 Stack Overflow Developer Survey Results Are InStandard errors for lasso prediction using RInference after using Lasso for variable selectionWhy is feature selection important, for classification tasks?Building reliable glmnet model and constructing predictionsHow can I implement lasso in R using optim functionR - Lasso Regression - different Lambda per regressorHow to interpret all zero coefficients in the results of cv.glmnet?Nested cross-validation with LASSO for model selection and evaluationglmnet returning lambda that gives all-zero coefficients as optimal lambdaLASSO: optimal $lambda$ drops all predictors from modelInference after using Lasso for variable selectionBuilding final model in glmnet after cross validationCOX model with Lasso using one dataset and predicting in a different dataset

Resizing object distorts it (Illustrator CC 2018)

Why did Acorn's A3000 have red function keys?

A poker game description that does not feel gimmicky

How to type this arrow in math mode?

Return to UK after being refused entry years previously

If a Druid sees an animal’s corpse, can they Wild Shape into that animal?

What do hard-Brexiteers want with respect to the Irish border?

Why do UK politicians seemingly ignore opinion polls on Brexit?

What are the motivations for publishing new editions of an existing textbook, beyond new discoveries in a field?

Which Sci-Fi work first showed weapon of galactic-scale mass destruction?

Can we generate random numbers using irrational numbers like π and e?

Where to refill my bottle in India?

What is the motivation for a law requiring 2 parties to consent for recording a conversation

What is the meaning of Triage in Cybersec world?

Is flight data recorder erased after every flight?

What is the meaning of the verb "bear" in this context?

Why didn't the Event Horizon Telescope team mention Sagittarius A*?

Are spiders unable to hurt humans, especially very small spiders?

slides for 30min~1hr skype tenure track application interview

What does Linus Torvalds mean when he says that Git "never ever" tracks a file?

How to deal with fear of taking dependencies

Multiply Two Integer Polynomials

Have you ever entered Singapore using a different passport or name?

The difference between dialogue marks



How to obtain Confidence Intervals for a LASSO regression?



The 2019 Stack Overflow Developer Survey Results Are InStandard errors for lasso prediction using RInference after using Lasso for variable selectionWhy is feature selection important, for classification tasks?Building reliable glmnet model and constructing predictionsHow can I implement lasso in R using optim functionR - Lasso Regression - different Lambda per regressorHow to interpret all zero coefficients in the results of cv.glmnet?Nested cross-validation with LASSO for model selection and evaluationglmnet returning lambda that gives all-zero coefficients as optimal lambdaLASSO: optimal $lambda$ drops all predictors from modelInference after using Lasso for variable selectionBuilding final model in glmnet after cross validationCOX model with Lasso using one dataset and predicting in a different dataset



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








2












$begingroup$


I'm very new from R. I have this code for a LASSO regression:



X <- X <- as.matrix(read.csv2("DB_LASSO_ERP.csv"))
y <- read.csv2("OUTCOME_LASSO_ERP.csv",header=F)$V1
fit <- glmnet(x = X, y = y, family = "binomial", alpha = 1)
crossval <- cv.glmnet(x = X, y = y, family = "binomial")
penalty <- crossval$
lambda.min
fit1 <- glmnet(x = X, y = y, family = "binomial", alpha = 1, lambda = penalty)


I want to obtain Confidence Intervals for this coefficients. How can I do? Can you help me with the script please? I have very few experience with R.
Thanks!










share|cite|improve this question







New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$







  • 1




    $begingroup$
    The answer here suggests that there is no consensus on how to calculate the standard errors of LASSO. Since you need the standard errors for confidence interval, you have to be very careful
    $endgroup$
    – V. Aslanyan
    11 hours ago






  • 1




    $begingroup$
    The link provided by @V.Aslanyan is quite useful, but note that the initial discussion on that page (from 2014) pre-dated much subsequent work on this topic.
    $endgroup$
    – EdM
    11 hours ago

















2












$begingroup$


I'm very new from R. I have this code for a LASSO regression:



X <- X <- as.matrix(read.csv2("DB_LASSO_ERP.csv"))
y <- read.csv2("OUTCOME_LASSO_ERP.csv",header=F)$V1
fit <- glmnet(x = X, y = y, family = "binomial", alpha = 1)
crossval <- cv.glmnet(x = X, y = y, family = "binomial")
penalty <- crossval$
lambda.min
fit1 <- glmnet(x = X, y = y, family = "binomial", alpha = 1, lambda = penalty)


I want to obtain Confidence Intervals for this coefficients. How can I do? Can you help me with the script please? I have very few experience with R.
Thanks!










share|cite|improve this question







New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$







  • 1




    $begingroup$
    The answer here suggests that there is no consensus on how to calculate the standard errors of LASSO. Since you need the standard errors for confidence interval, you have to be very careful
    $endgroup$
    – V. Aslanyan
    11 hours ago






  • 1




    $begingroup$
    The link provided by @V.Aslanyan is quite useful, but note that the initial discussion on that page (from 2014) pre-dated much subsequent work on this topic.
    $endgroup$
    – EdM
    11 hours ago













2












2








2


1



$begingroup$


I'm very new from R. I have this code for a LASSO regression:



X <- X <- as.matrix(read.csv2("DB_LASSO_ERP.csv"))
y <- read.csv2("OUTCOME_LASSO_ERP.csv",header=F)$V1
fit <- glmnet(x = X, y = y, family = "binomial", alpha = 1)
crossval <- cv.glmnet(x = X, y = y, family = "binomial")
penalty <- crossval$
lambda.min
fit1 <- glmnet(x = X, y = y, family = "binomial", alpha = 1, lambda = penalty)


I want to obtain Confidence Intervals for this coefficients. How can I do? Can you help me with the script please? I have very few experience with R.
Thanks!










share|cite|improve this question







New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$




I'm very new from R. I have this code for a LASSO regression:



X <- X <- as.matrix(read.csv2("DB_LASSO_ERP.csv"))
y <- read.csv2("OUTCOME_LASSO_ERP.csv",header=F)$V1
fit <- glmnet(x = X, y = y, family = "binomial", alpha = 1)
crossval <- cv.glmnet(x = X, y = y, family = "binomial")
penalty <- crossval$
lambda.min
fit1 <- glmnet(x = X, y = y, family = "binomial", alpha = 1, lambda = penalty)


I want to obtain Confidence Intervals for this coefficients. How can I do? Can you help me with the script please? I have very few experience with R.
Thanks!







regression confidence-interval lasso glmnet






share|cite|improve this question







New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|cite|improve this question







New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|cite|improve this question




share|cite|improve this question






New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 12 hours ago









AlfonsoAlfonso

111




111




New contributor




Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Alfonso is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







  • 1




    $begingroup$
    The answer here suggests that there is no consensus on how to calculate the standard errors of LASSO. Since you need the standard errors for confidence interval, you have to be very careful
    $endgroup$
    – V. Aslanyan
    11 hours ago






  • 1




    $begingroup$
    The link provided by @V.Aslanyan is quite useful, but note that the initial discussion on that page (from 2014) pre-dated much subsequent work on this topic.
    $endgroup$
    – EdM
    11 hours ago












  • 1




    $begingroup$
    The answer here suggests that there is no consensus on how to calculate the standard errors of LASSO. Since you need the standard errors for confidence interval, you have to be very careful
    $endgroup$
    – V. Aslanyan
    11 hours ago






  • 1




    $begingroup$
    The link provided by @V.Aslanyan is quite useful, but note that the initial discussion on that page (from 2014) pre-dated much subsequent work on this topic.
    $endgroup$
    – EdM
    11 hours ago







1




1




$begingroup$
The answer here suggests that there is no consensus on how to calculate the standard errors of LASSO. Since you need the standard errors for confidence interval, you have to be very careful
$endgroup$
– V. Aslanyan
11 hours ago




$begingroup$
The answer here suggests that there is no consensus on how to calculate the standard errors of LASSO. Since you need the standard errors for confidence interval, you have to be very careful
$endgroup$
– V. Aslanyan
11 hours ago




1




1




$begingroup$
The link provided by @V.Aslanyan is quite useful, but note that the initial discussion on that page (from 2014) pre-dated much subsequent work on this topic.
$endgroup$
– EdM
11 hours ago




$begingroup$
The link provided by @V.Aslanyan is quite useful, but note that the initial discussion on that page (from 2014) pre-dated much subsequent work on this topic.
$endgroup$
– EdM
11 hours ago










1 Answer
1






active

oldest

votes


















6












$begingroup$

Please think very carefully about why you want confidence intervals for the LASSO coefficients and how you will interpret them. This is not an easy problem.



The predictors chosen by LASSO (as for any feature-selection method) can be highly dependent on the data sample at hand. You can examine this in your own data by repeating your LASSO model-building procedure on multiple bootstrap samples of the data. If you have predictors that are correlated with each other, the specific predictors chosen by LASSO are likely to differ among models based on the different bootstrap samples. So what do you mean by a confidence interval for a coefficient for a predictor, say predictor $x_1$, if $x_1$ wouldn't even have been chosen by LASSO if you had worked with a different sample from the same population?



Despite this instability in feature selection, LASSO-based models can be useful for prediction. The selection of 1 from among several correlated predictors might be somewhat arbitrary, but the 1 selected serves as a rough proxy for the others and thus can lead to valid predictions. The quality of predictions from a LASSO model is typically of more interest than are confidence intervals for the coefficients. You can test the performance of your LASSO approach by seeing how well the models based on multiple bootstrapped samples work on the full original data set.



That said, there is recent work on principled ways to obtain confidence intervals and on related issues in inference after LASSO. This page and its links is a good place to start. The issues are discussed in more detail in Section 6.3 of Statistical Learning with Sparsity. There is also a package selectiveInference in R that implements these methods. But these are based on specific assumptions that might not hold in your data. If you do choose to use this approach, make sure to understand the conditions under which the approach is valid and exactly what those confidence intervals really mean. That statistical issue, rather than the R coding issue, is what is crucial here.






share|cite|improve this answer











$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "65"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );






    Alfonso is a new contributor. Be nice, and check out our Code of Conduct.









    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f402267%2fhow-to-obtain-confidence-intervals-for-a-lasso-regression%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    6












    $begingroup$

    Please think very carefully about why you want confidence intervals for the LASSO coefficients and how you will interpret them. This is not an easy problem.



    The predictors chosen by LASSO (as for any feature-selection method) can be highly dependent on the data sample at hand. You can examine this in your own data by repeating your LASSO model-building procedure on multiple bootstrap samples of the data. If you have predictors that are correlated with each other, the specific predictors chosen by LASSO are likely to differ among models based on the different bootstrap samples. So what do you mean by a confidence interval for a coefficient for a predictor, say predictor $x_1$, if $x_1$ wouldn't even have been chosen by LASSO if you had worked with a different sample from the same population?



    Despite this instability in feature selection, LASSO-based models can be useful for prediction. The selection of 1 from among several correlated predictors might be somewhat arbitrary, but the 1 selected serves as a rough proxy for the others and thus can lead to valid predictions. The quality of predictions from a LASSO model is typically of more interest than are confidence intervals for the coefficients. You can test the performance of your LASSO approach by seeing how well the models based on multiple bootstrapped samples work on the full original data set.



    That said, there is recent work on principled ways to obtain confidence intervals and on related issues in inference after LASSO. This page and its links is a good place to start. The issues are discussed in more detail in Section 6.3 of Statistical Learning with Sparsity. There is also a package selectiveInference in R that implements these methods. But these are based on specific assumptions that might not hold in your data. If you do choose to use this approach, make sure to understand the conditions under which the approach is valid and exactly what those confidence intervals really mean. That statistical issue, rather than the R coding issue, is what is crucial here.






    share|cite|improve this answer











    $endgroup$

















      6












      $begingroup$

      Please think very carefully about why you want confidence intervals for the LASSO coefficients and how you will interpret them. This is not an easy problem.



      The predictors chosen by LASSO (as for any feature-selection method) can be highly dependent on the data sample at hand. You can examine this in your own data by repeating your LASSO model-building procedure on multiple bootstrap samples of the data. If you have predictors that are correlated with each other, the specific predictors chosen by LASSO are likely to differ among models based on the different bootstrap samples. So what do you mean by a confidence interval for a coefficient for a predictor, say predictor $x_1$, if $x_1$ wouldn't even have been chosen by LASSO if you had worked with a different sample from the same population?



      Despite this instability in feature selection, LASSO-based models can be useful for prediction. The selection of 1 from among several correlated predictors might be somewhat arbitrary, but the 1 selected serves as a rough proxy for the others and thus can lead to valid predictions. The quality of predictions from a LASSO model is typically of more interest than are confidence intervals for the coefficients. You can test the performance of your LASSO approach by seeing how well the models based on multiple bootstrapped samples work on the full original data set.



      That said, there is recent work on principled ways to obtain confidence intervals and on related issues in inference after LASSO. This page and its links is a good place to start. The issues are discussed in more detail in Section 6.3 of Statistical Learning with Sparsity. There is also a package selectiveInference in R that implements these methods. But these are based on specific assumptions that might not hold in your data. If you do choose to use this approach, make sure to understand the conditions under which the approach is valid and exactly what those confidence intervals really mean. That statistical issue, rather than the R coding issue, is what is crucial here.






      share|cite|improve this answer











      $endgroup$















        6












        6








        6





        $begingroup$

        Please think very carefully about why you want confidence intervals for the LASSO coefficients and how you will interpret them. This is not an easy problem.



        The predictors chosen by LASSO (as for any feature-selection method) can be highly dependent on the data sample at hand. You can examine this in your own data by repeating your LASSO model-building procedure on multiple bootstrap samples of the data. If you have predictors that are correlated with each other, the specific predictors chosen by LASSO are likely to differ among models based on the different bootstrap samples. So what do you mean by a confidence interval for a coefficient for a predictor, say predictor $x_1$, if $x_1$ wouldn't even have been chosen by LASSO if you had worked with a different sample from the same population?



        Despite this instability in feature selection, LASSO-based models can be useful for prediction. The selection of 1 from among several correlated predictors might be somewhat arbitrary, but the 1 selected serves as a rough proxy for the others and thus can lead to valid predictions. The quality of predictions from a LASSO model is typically of more interest than are confidence intervals for the coefficients. You can test the performance of your LASSO approach by seeing how well the models based on multiple bootstrapped samples work on the full original data set.



        That said, there is recent work on principled ways to obtain confidence intervals and on related issues in inference after LASSO. This page and its links is a good place to start. The issues are discussed in more detail in Section 6.3 of Statistical Learning with Sparsity. There is also a package selectiveInference in R that implements these methods. But these are based on specific assumptions that might not hold in your data. If you do choose to use this approach, make sure to understand the conditions under which the approach is valid and exactly what those confidence intervals really mean. That statistical issue, rather than the R coding issue, is what is crucial here.






        share|cite|improve this answer











        $endgroup$



        Please think very carefully about why you want confidence intervals for the LASSO coefficients and how you will interpret them. This is not an easy problem.



        The predictors chosen by LASSO (as for any feature-selection method) can be highly dependent on the data sample at hand. You can examine this in your own data by repeating your LASSO model-building procedure on multiple bootstrap samples of the data. If you have predictors that are correlated with each other, the specific predictors chosen by LASSO are likely to differ among models based on the different bootstrap samples. So what do you mean by a confidence interval for a coefficient for a predictor, say predictor $x_1$, if $x_1$ wouldn't even have been chosen by LASSO if you had worked with a different sample from the same population?



        Despite this instability in feature selection, LASSO-based models can be useful for prediction. The selection of 1 from among several correlated predictors might be somewhat arbitrary, but the 1 selected serves as a rough proxy for the others and thus can lead to valid predictions. The quality of predictions from a LASSO model is typically of more interest than are confidence intervals for the coefficients. You can test the performance of your LASSO approach by seeing how well the models based on multiple bootstrapped samples work on the full original data set.



        That said, there is recent work on principled ways to obtain confidence intervals and on related issues in inference after LASSO. This page and its links is a good place to start. The issues are discussed in more detail in Section 6.3 of Statistical Learning with Sparsity. There is also a package selectiveInference in R that implements these methods. But these are based on specific assumptions that might not hold in your data. If you do choose to use this approach, make sure to understand the conditions under which the approach is valid and exactly what those confidence intervals really mean. That statistical issue, rather than the R coding issue, is what is crucial here.







        share|cite|improve this answer














        share|cite|improve this answer



        share|cite|improve this answer








        edited 11 hours ago

























        answered 11 hours ago









        EdMEdM

        22.3k23496




        22.3k23496




















            Alfonso is a new contributor. Be nice, and check out our Code of Conduct.









            draft saved

            draft discarded


















            Alfonso is a new contributor. Be nice, and check out our Code of Conduct.












            Alfonso is a new contributor. Be nice, and check out our Code of Conduct.











            Alfonso is a new contributor. Be nice, and check out our Code of Conduct.














            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f402267%2fhow-to-obtain-confidence-intervals-for-a-lasso-regression%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Reverse int within the 32-bit signed integer range: [−2^31, 2^31 − 1]Combining two 32-bit integers into one 64-bit integerDetermine if an int is within rangeLossy packing 32 bit integer to 16 bitComputing the square root of a 64-bit integerKeeping integer addition within boundsSafe multiplication of two 64-bit signed integersLeetcode 10: Regular Expression MatchingSigned integer-to-ascii x86_64 assembler macroReverse the digits of an Integer“Add two numbers given in reverse order from a linked list”

            Category:Fedor von Bock Media in category "Fedor von Bock"Navigation menuUpload mediaISNI: 0000 0000 5511 3417VIAF ID: 24712551GND ID: 119294796Library of Congress authority ID: n96068363BnF ID: 12534305fSUDOC authorities ID: 034604189Open Library ID: OL338253ANKCR AUT ID: jn19990000869National Library of Israel ID: 000514068National Thesaurus for Author Names ID: 341574317ReasonatorScholiaStatistics

            Kiel Indholdsfortegnelse Historie | Transport og færgeforbindelser | Sejlsport og anden sport | Kultur | Kendte personer fra Kiel | Noter | Litteratur | Eksterne henvisninger | Navigationsmenuwww.kiel.de54°19′31″N 10°8′26″Ø / 54.32528°N 10.14056°Ø / 54.32528; 10.14056Oberbürgermeister Dr. Ulf Kämpferwww.statistik-nord.deDen danske Stats StatistikKiels hjemmesiderrrWorldCat312794080n790547494030481-4