Why Normality assumption in linear regression












2












$begingroup$


My question is very simple: why we choose normal as the distribution that error term follows in the assumption of linear regression? Why we don't choose others like uniform, t or whatever?






regression mathematical-statistics normal-distribution error linear







share|cite|improve this question









$endgroup$












  • $begingroup$
    We don't choose the normal assumption. It just happens to be the case that when the error is normal, the model coefficients exactly follow a normal distribution and an exact F-test can be used to test hypotheses about them.
    $endgroup$
    – AdamO
    2 hours ago












  • $begingroup$
    Because the math works out easily enough that people could use it before modern computers.
    $endgroup$
    – Nat
    2 hours ago


















2












$begingroup$


My question is very simple: why we choose normal as the distribution that error term follows in the assumption of linear regression? Why we don't choose others like uniform, t or whatever?






regression mathematical-statistics normal-distribution error linear







share|cite|improve this question









$endgroup$












  • $begingroup$
    We don't choose the normal assumption. It just happens to be the case that when the error is normal, the model coefficients exactly follow a normal distribution and an exact F-test can be used to test hypotheses about them.
    $endgroup$
    – AdamO
    2 hours ago












  • $begingroup$
    Because the math works out easily enough that people could use it before modern computers.
    $endgroup$
    – Nat
    2 hours ago
















2












2








2





$begingroup$


My question is very simple: why we choose normal as the distribution that error term follows in the assumption of linear regression? Why we don't choose others like uniform, t or whatever?






regression mathematical-statistics normal-distribution error linear







share|cite|improve this question









$endgroup$




My question is very simple: why we choose normal as the distribution that error term follows in the assumption of linear regression? Why we don't choose others like uniform, t or whatever?






regression mathematical-statistics normal-distribution error linear




regression mathematical-statistics normal-distribution error linear






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked 3 hours ago









Master ShiMaster Shi

211




211












  • $begingroup$
    We don't choose the normal assumption. It just happens to be the case that when the error is normal, the model coefficients exactly follow a normal distribution and an exact F-test can be used to test hypotheses about them.
    $endgroup$
    – AdamO
    2 hours ago












  • $begingroup$
    Because the math works out easily enough that people could use it before modern computers.
    $endgroup$
    – Nat
    2 hours ago




















  • $begingroup$
    We don't choose the normal assumption. It just happens to be the case that when the error is normal, the model coefficients exactly follow a normal distribution and an exact F-test can be used to test hypotheses about them.
    $endgroup$
    – AdamO
    2 hours ago












  • $begingroup$
    Because the math works out easily enough that people could use it before modern computers.
    $endgroup$
    – Nat
    2 hours ago


















$begingroup$
We don't choose the normal assumption. It just happens to be the case that when the error is normal, the model coefficients exactly follow a normal distribution and an exact F-test can be used to test hypotheses about them.
$endgroup$
– AdamO
2 hours ago






$begingroup$
We don't choose the normal assumption. It just happens to be the case that when the error is normal, the model coefficients exactly follow a normal distribution and an exact F-test can be used to test hypotheses about them.
$endgroup$
– AdamO
2 hours ago














$begingroup$
Because the math works out easily enough that people could use it before modern computers.
$endgroup$
– Nat
2 hours ago






$begingroup$
Because the math works out easily enough that people could use it before modern computers.
$endgroup$
– Nat
2 hours ago












1 Answer
1






active

oldest

votes


















6












$begingroup$

You can choose another error distribution; they basically just change the loss function.



This is certainly done.



Laplace (double exponential errors) correspond to least absolute deviations regression/$L_1$ regression (which numerous posts on site discuss). Regressions with t-errors are occasionally used (in some cases because they're more robust to gross errors), though they can have a disadvantage -- the likelihood (and therefore the negative of the loss) can have multiple modes.



Uniform errors correspond to an $L_infty$ loss (minimize the maximum deviation); such regression is sometimes called Chebyshev approximation (though beware, since there's another thing with essentially the same name). Again, this is sometimes done (indeed for simple regression and smallish data sets with bounded errors with constant spread the fit is often easy enough to find by hand, directly on a plot, though in practice you can use linear programming methods, or other algorithms; indeed, $L_infty$ and $L_1$ regression problems are duals of each other, which can lead to sometimes convenient shortcuts for some problems).



Many other choices are possible and quite a few have been used in practice.



[Note that if you have additive, independent, constant-spread errors with a density of the form $k,exp(-c.g(varepsilon))$, maximizing the likelihood will correspond to minimizing $sum_i g(e_i)$, where $e_i$ is the $i$th residual.]






share|cite|improve this answer











$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "65"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f395011%2fwhy-normality-assumption-in-linear-regression%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    6












    $begingroup$

    You can choose another error distribution; they basically just change the loss function.



    This is certainly done.



    Laplace (double exponential errors) correspond to least absolute deviations regression/$L_1$ regression (which numerous posts on site discuss). Regressions with t-errors are occasionally used (in some cases because they're more robust to gross errors), though they can have a disadvantage -- the likelihood (and therefore the negative of the loss) can have multiple modes.



    Uniform errors correspond to an $L_infty$ loss (minimize the maximum deviation); such regression is sometimes called Chebyshev approximation (though beware, since there's another thing with essentially the same name). Again, this is sometimes done (indeed for simple regression and smallish data sets with bounded errors with constant spread the fit is often easy enough to find by hand, directly on a plot, though in practice you can use linear programming methods, or other algorithms; indeed, $L_infty$ and $L_1$ regression problems are duals of each other, which can lead to sometimes convenient shortcuts for some problems).



    Many other choices are possible and quite a few have been used in practice.



    [Note that if you have additive, independent, constant-spread errors with a density of the form $k,exp(-c.g(varepsilon))$, maximizing the likelihood will correspond to minimizing $sum_i g(e_i)$, where $e_i$ is the $i$th residual.]






    share|cite|improve this answer











    $endgroup$


















      6












      $begingroup$

      You can choose another error distribution; they basically just change the loss function.



      This is certainly done.



      Laplace (double exponential errors) correspond to least absolute deviations regression/$L_1$ regression (which numerous posts on site discuss). Regressions with t-errors are occasionally used (in some cases because they're more robust to gross errors), though they can have a disadvantage -- the likelihood (and therefore the negative of the loss) can have multiple modes.



      Uniform errors correspond to an $L_infty$ loss (minimize the maximum deviation); such regression is sometimes called Chebyshev approximation (though beware, since there's another thing with essentially the same name). Again, this is sometimes done (indeed for simple regression and smallish data sets with bounded errors with constant spread the fit is often easy enough to find by hand, directly on a plot, though in practice you can use linear programming methods, or other algorithms; indeed, $L_infty$ and $L_1$ regression problems are duals of each other, which can lead to sometimes convenient shortcuts for some problems).



      Many other choices are possible and quite a few have been used in practice.



      [Note that if you have additive, independent, constant-spread errors with a density of the form $k,exp(-c.g(varepsilon))$, maximizing the likelihood will correspond to minimizing $sum_i g(e_i)$, where $e_i$ is the $i$th residual.]






      share|cite|improve this answer











      $endgroup$
















        6












        6








        6





        $begingroup$

        You can choose another error distribution; they basically just change the loss function.



        This is certainly done.



        Laplace (double exponential errors) correspond to least absolute deviations regression/$L_1$ regression (which numerous posts on site discuss). Regressions with t-errors are occasionally used (in some cases because they're more robust to gross errors), though they can have a disadvantage -- the likelihood (and therefore the negative of the loss) can have multiple modes.



        Uniform errors correspond to an $L_infty$ loss (minimize the maximum deviation); such regression is sometimes called Chebyshev approximation (though beware, since there's another thing with essentially the same name). Again, this is sometimes done (indeed for simple regression and smallish data sets with bounded errors with constant spread the fit is often easy enough to find by hand, directly on a plot, though in practice you can use linear programming methods, or other algorithms; indeed, $L_infty$ and $L_1$ regression problems are duals of each other, which can lead to sometimes convenient shortcuts for some problems).



        Many other choices are possible and quite a few have been used in practice.



        [Note that if you have additive, independent, constant-spread errors with a density of the form $k,exp(-c.g(varepsilon))$, maximizing the likelihood will correspond to minimizing $sum_i g(e_i)$, where $e_i$ is the $i$th residual.]






        share|cite|improve this answer











        $endgroup$



        You can choose another error distribution; they basically just change the loss function.



        This is certainly done.



        Laplace (double exponential errors) correspond to least absolute deviations regression/$L_1$ regression (which numerous posts on site discuss). Regressions with t-errors are occasionally used (in some cases because they're more robust to gross errors), though they can have a disadvantage -- the likelihood (and therefore the negative of the loss) can have multiple modes.



        Uniform errors correspond to an $L_infty$ loss (minimize the maximum deviation); such regression is sometimes called Chebyshev approximation (though beware, since there's another thing with essentially the same name). Again, this is sometimes done (indeed for simple regression and smallish data sets with bounded errors with constant spread the fit is often easy enough to find by hand, directly on a plot, though in practice you can use linear programming methods, or other algorithms; indeed, $L_infty$ and $L_1$ regression problems are duals of each other, which can lead to sometimes convenient shortcuts for some problems).



        Many other choices are possible and quite a few have been used in practice.



        [Note that if you have additive, independent, constant-spread errors with a density of the form $k,exp(-c.g(varepsilon))$, maximizing the likelihood will correspond to minimizing $sum_i g(e_i)$, where $e_i$ is the $i$th residual.]







        share|cite|improve this answer














        share|cite|improve this answer



        share|cite|improve this answer








        edited 59 mins ago

























        answered 2 hours ago









        Glen_b♦Glen_b

        212k22409758




        212k22409758






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f395011%2fwhy-normality-assumption-in-linear-regression%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            What other Star Trek series did the main TNG cast show up in?

            Image
            .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0; } 4 I'm referring to Picard, Riker, Data, Worf, Geordi, Dr. Crusher, Wesley Crusher, Chief O'Brien and family, Deanna Troi, Guinan. I know Picard and Worf show up on DS9, though I haven't seen it myself. I'm curious what happened to the characters after that last episode of TNG? star-trek star-trek-tng share | improve this question edited 10 hours ago Ham Sandwich 6,051 2 25 75 ...
            Read more

            Berlina muro

            Image
            La berlina muro ĉe Bethaniendamm, 1986 La berlina muro aŭ simple la muro [1] , nomita ankaŭ "kontraŭfaŝisma remparo" en propaganda lingvo de la iama Germana Demokratia Respubliko, estis parto de la interngermana limo, kiu dividis de la 13-a de aŭgusto 1961 ĝis la 9-a de novembro 1989 okcidentan Berlinon de la orienta parto de la urbo kaj de la ĉirkaŭa tereno de la GDR. Ĝi estis unu el la plej famaj simboloj de la malvarma milito kaj de la disdivido de Germanio. Pli ol simpla muro, la berlina muro estis fakte tuta defendokonstruaĵo kun du muroj, ena malplena strio, rondirvojo, gardturoj kaj alarmsistemoj. Dum la provo transiri la severe garditan limon al okcidenta Berlino multaj homoj estis mortigitaj. La preciza nombro de viktimoj estas pridisputita kaj ne certa; la nombroj anoncataj varias inter 86 kaj 238 mortintoj. La malplifortiĝo de Sovetunio kaj la politiko de liberigo gvidita de Miĥail Gorbaĉov ebligis al la orientgermanoj faligi la 9-an de novembro 1989 l...
            Read more

            Berlina aerponto

            Image
            Flugkoridoroj dum la blokado de Berlino La Berlina aerponto estas la provizigo de okcidenta Berlino per aviadiloj fare de la aliancanoj inter la 23-a de junio 1948 kaj la 12-a de majo 1949 dum la blokado de Berlino fare de la sovetia armeo. Enhavo 1 Historio 1.1 La aerkoridoroj 1.2 Unuaj flugoj 1.3 Plibonigo de la aerponto 1.4 Fino de la aerponto 1.5 Rozinbombiloj 1.6 Bilanco 2 Flughavenoj 2.1 Berlino 2.2 Okcidenta Germanio 3 Aviadiloj de la berlina aerponto 4 15-a datreveno de la aerponto 5 Bibliografio 6 Referencoj 7 Eksteraj ligiloj Historio | General Lucius D. Clay, militguberniestro de la usona okupadzono Jam antaŭ la berlina aerponto estiĝis la malgranda aerponto . La estro de la sovetia militadministracio en Germanio, Vasili Daniloviĉ Sokolovski, ordonis la 1-an de aprilo 1948, kiel reago al la konferenco de Londono inter britoj, francoj kaj reprezentantoj de Benelukso pri aligo de Germa...
            Read more