›Linear Regression Simulator Back
Statistics

Linear Regression Simulator

Generate random data by changing the noise, true slope and number of points, and watch the line approach the least-squares solution. See the coefficient of determination R², residuals and correlation coefficient in real time to get a feel for how least squares works.

Parameters

While paused, moving a slider updates the results immediately.

Results
—
Slope b
—
Intercept a
—
Coefficient of determination R²
—
Sum of squared residuals SSR
—
Correlation coefficient r
0
Number of points n
y = — x + —
Data points and regression line
Theory & Key Formulas
\(b = \frac{\sum(x_i-\bar{x})(y_i-\bar{y})}{\sum(x_i-\bar{x})^2}\)
\(a = \bar{y} - b\bar{x}\)
\(R^2 = 1 - \frac{SS_E}{SS_T} = \frac{SS_R}{SS_T}\)

💬 Explanation dialogue

🙋
Is linear regression basically "drawing the line closest to the points"? But how do you measure "closest"?
🎓
The least-squares method finds "the line that minimises the sum of the squared vertical distances (residuals)". Why squared? So that positive and negative residuals do not cancel out, and so that large deviations count more heavily. Turn on "Show residual lines" in this tool to see the vertical distance from each point to the line.
🙋
So does an R² of 0.8 mean "a fairly good fit"?
🎓
Yes. It means "the model explains 80% of the variation in the data". But the standard varies a lot by field: in a physics experiment anything below R² = 0.99 may be disappointing, while for economic data 0.5 can still be meaningful. You cannot simply say "higher is better".
🙋
Can a single outlier in the data shift the line a lot? How do you deal with that?
🎓
It can. Because the residuals are squared and summed, one far-away point pulls the line strongly towards itself. The first step is to find out why the outlier occurred: if it is a measurement error you can exclude it, but if it is a real phenomenon you cannot. Statistically, you can use robust regression (such as L1 regression, which minimises absolute values). In machine learning the Huber loss function is well known.

Frequently Asked Questions

Q. What is the difference between correlation and causation?
A. Correlation means two variables vary together; causation is a cause-and-effect relationship. "Days when more ice cream is sold have more drowning accidents" is a correlation, but ice cream does not cause drowning (summer is the common cause). Regression analysis quantifies correlation but does not prove causation.
Q. What is the difference between simple and multiple regression?
A. This tool performs simple regression with one explanatory variable (y = a + bx). Multiple regression uses several explanatory variables (y = a + b₁x₁ + b₂x₂ + …). In CAE, multivariable regression such as "predicting stress from material properties, load and temperature" is common.
Q. What is the difference between a confidence interval and a prediction interval?
A. A confidence interval is the range of uncertainty of the regression line (the mean), while a prediction interval is the range for the predicted value of one new point. The prediction interval is always wider. This tool does not draw a confidence interval; it shows the least-squares line together with the coefficient of determination R², the sum of squared residuals SSR and the correlation coefficient r.
Q. How does it differ from machine learning?
A. Linear regression is the foundation of machine learning and the simplest supervised learning model. Machine learning also uses linear regression with regularisation (LASSO, Ridge) and non-linear models (decision trees, neural networks). Classical regression emphasises statistical interpretation (p-values, confidence intervals), while the machine-learning approach emphasises prediction accuracy.

What Is the Linear Regression and Least Squares Simulator?

The model of linear regression assumes a straight line \(y = a + b x\) (a: intercept, b: slope) for the observed data points \((x_i, y_i)\). The least-squares method finds the parameters \(a, b\) that minimise the sum of squares \(S = \sum_{i=1}^{n} \varepsilon_i^2\) of the vertical errors (residuals) \(\varepsilon_i = y_i - (a + b x_i)\). Solving \(\frac{\partial S}{\partial a} = 0\) and \(\frac{\partial S}{\partial b} = 0\) gives the analytical result \(b = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}\), \(a = \bar{y} - b \bar{x}\), where \(\bar{x}, \bar{y}\) are the means. The coefficient of determination \(R^2 = 1 - \frac{\sum \varepsilon_i^2}{\sum (y_i - \bar{y})^2}\) expresses the goodness of fit as a value from 0 to 1 and, for simple regression, equals the square of the correlation coefficient r. In this simulator the data are regenerated every time you change the noise, true slope or number of points, and you can watch in real time as the line approaches the least-squares solution and the residuals and R² change. This gives an intuitive feel for how least squares captures the scatter in the data and determines the line.

Real-World Applications

Industrial use
In manufacturing, linear regression is widely used to analyse component durability test data. For example, the relationship between temperature and fatigue life is modelled by least squares and the fit is assessed with R², helping to plan prototypes and additional tests. In semiconductor manufacturing, regression is also used to capture the relationship between equipment parameters such as gas flow rate and results such as film thickness, and to tune process conditions.

Research and education
In university physics labs, students plot spring extension against the mass of the weights and verify Hooke's law by linear regression. Turning on the residual lines in this tool shows visually what least squares is minimising. In environmental science, it is also used as teaching material for analysing long-term changes in temperature or carbon dioxide concentration and discussing the validity of a model from R².

Relation to CAE analysis and practical use
When validating CAE simulation results, linear regression is used to assess the correlation between experimental data and analysis values. For example, you regress the stresses from an FEM analysis against measured values and check whether the slope is close to 1, the intercept close to 0 and R² high enough (even with a high R², a slope that departs from 1 indicates a systematic error). This tool is a learning aid for checking that idea quickly.

Common Misconceptions and Points to Note

It is tempting to think that "the more data points, the higher R² must be", but forcing a straight line through data that contain outliers or a non-linear relationship can lower R². R² shows the goodness of fit, but it depends heavily on the quality and distribution of the data, so simply adding samples does not necessarily improve it.

A common misunderstanding is that "all data points should fall inside the 95% confidence interval". In fact the confidence interval expresses the uncertainty in the average position of the regression line, which is different from the prediction interval that describes the scatter of individual points. Many points lying outside the confidence interval is therefore not a statistical problem; take care not to confuse the two.

It is tempting to think that "least squares is a universal method that works for any data", but it is very sensitive to outliers: a single extreme point can distort the regression line badly. When the data are not linear or the errors are not normally distributed, other regression methods or data preprocessing should be considered.

How to Use

  1. Move the "Scatter σ (noise)", "True slope" and "Number of points n" sliders to generate random points around the true line y = (true slope) × x + 1.5
  2. Watch the blue line move from its horizontal starting position to the least-squares line. Turn on "Show residual lines" to see the vertical distance (residual) from each point to the line
  3. The coefficient of determination R², sum of squared residuals SSR, correlation coefficient r, slope b and intercept a update in real time. Turn on "Auto-sweep noise" to vary σ automatically and see R² fall continuously as the noise grows

Worked Example

The points in this tool are generated around the true line y = x + 1.5 (true slope 1, intercept 1.5), with x drawn uniformly from 0.4 to 9.6. The variance of x is 9.2²/12 ≈ 7.05, so a rule of thumb for the coefficient of determination at noise σ is R² ≈ 7.05/(7.05 + σ²): R² ≈ 0.97 for σ = 0.5, R² ≈ 0.78 for the initial σ = 1.4 and R² ≈ 0.44 for σ = 3. Because the points are regenerated randomly each time, the displayed value scatters around this estimate (about 0.70–0.85 measured with n = 24 and σ = 1.4). The scatter (standard error) of the slope b is about σ/√(n × 7.05), roughly 0.11 for σ = 1.4 and n = 24.

Practical Notes