Distinctive Analytics

A Place to enhance your analytical skills

ANOVA (Analysis of Variance ) part 1

Distinctive Analytics > Blog > Lessons > ANOVA (Analysis of Variance ) part 1

A method to find a statistical relationship between two variables in a dataset where one variable is used to group data.

Topics: , ,

Definition of ANOVA: A statistical method in which the variation in a set of observations is divided into distinct components. According to Oxford languages.

Simply put, it would be a method to find a statistical relationship between two variables in a dataset where one variable is used to group data.

The method to find the statistical significance is to calculate the variance across the whole dataset, the variance between groups, and the variance within groups.

To perform ANOVA on a dataset, we need one categorical and one continuous variable.

Types of ANOVA

  • One Way ANOVA
  • Two Way ANOVA

One Way ANOVA

One-way ANOVA is used to compare three or more groups based on one categorical and one continuous variable.

Import Required Libraries

Python
Python
Python
import pandas as pd
import numpy as np
import random
# Library to extract f-value
import scipy.stats

Create Dataset

Python
Python
Python
# create random x variable
x=random.sample(range(0, 16), 15)
data={'x':x,'y':[1,1,1,1,1,2,2,2,2,2,3,3,3,3,3]}
df=pd.DataFrame(data)
01234567891011121314151617
x234342202221202122353637363735
y111111222222333333
zaaabbbaaabbbaaabbb

Let’s start with grouping data. We will group X according to Y.

Python
Python
Python
for i in df['y'].unique():
    print(f'Group {i}')
    print(df[df['y']==i],end='\n\n')
Output
Group 1
   x  y  z
0  2  1  a
1  3  1  a
2  4  1  a
3  3  1  b
4  4  1  b
5  2  1  b

Group 2
     x  y  z
6   20  2  a
7   22  2  a
8   21  2  a
9   20  2  b
10  21  2  b
11  22  2  b

Group 3
     x  y  z
12  35  3  a
13  36  3  a
14  37  3  a
15  36  3  b
16  37  3  b
17  35  3  b
Python

Calculating the means of each group

Python
Python
Python
groupM=df.groupby('y').mean().reset_index()
groupM
Output
  y	 x
0	1	3.0
1	2	21.0
2	3	36.0
Python

Calculate the mean of all data (grand mean)

Python
Python
Python
grandM=df['x'].sum()/15
grandM
Output
20.0
Python

Calculate the Sum of Squares Total

image 13
Python
Python
Python
df['sst']=(df['x']-(df.x.sum()/len(df)))**2
SST=df['sst'].sum()
print(SST)
Output
3288.0
Python

Calculate the Sum of Squares Within

image 16
Python
Python
Python
SSW=0
for i in list(df['y'].unique()):
    #print(i)
    g=pd.DataFrame(df[df['y']==i].x-float(groupM[groupM['y']==i].x))**2
    SSW+=g.sum()
print(float(SSW))
Output
8490.0
Python

Calculate Sum of Squares Between

Python
Python
Python
groupM['ssb']=groupM.x-grandM
groupM['ssb']=groupM['ssb']**2 
groupM['ssb']=groupM['ssb']*N
SSB=groupM.ssb.sum()
print(SSB)
Output
2730.0
Python

All the knowledge that has been obtained up to this point is used to comprehend the occasion, setting, and context as well as the meaning of the statement.

Calculate the Degree of freedom for SST, SSW, SSB

The concept of degree of freedom is a method for comprehending logically independent values. By calculating the degree of freedom, we can get the scope of an interpretable sample of factors in the dataset.

Python
Python
Python
N=len((df[df['y']==1]))
M=len(df['y'].unique())
print(M,N)
Output
3 6
Python
Python
Python
Python
# Degrees of Freedom for all 
sstdf= (M*N)-1
sswdf= M*(N-1)
ssbdf= M-1

print(f'DF\n SST= {sstdf}\n SSW= {sswdf}\n SSB= {ssbdf}')
Output
DF
 SST= 17
 SSW= 15
 SSB= 2
Python

Values Table

Let’s create a table with all the values we have found so far.

Python
Python
Python
final_table=pd.DataFrame({
    'Sum of Squares':[float(SSW),float(SSB),float(SST),np.nan],
    'Degree of Freedom':[sswdf,ssbdf,sstdf,np.nan],
    'Mean Square':[np.nan for x in range(4)],
    'F score':[np.nan for x in range(4)],
    'F Value':[np.nan for x in range(4)],
    'H0':[np.nan for x in range(4)]} ,
    index=['Sum of Squares Within','Sum of Squares Between','Sum of Squares Total','Result'])
final_table
Sum of SquaresDegree of FreedomMean SquareF scoreF ValueH0
Sum of Squares Within220.812.0
Sum of Squares Between59.22.0
Sum of Squares Total280.014.0
Result

Mean Square

To calculate the mean square, we divide the degree of freedom by the respective sum of squares.

image 20
Python
Python
Python
final_table.loc[:'Sum of Squares Between']['Mean Square']=final_table.loc[:'Sum of Squares Between']['Sum of Squares']/final_table.loc[:'Sum of Squares Between']['Degree of Freedom']
final_table
Sum of SquaresDegree of FreedomMean SquareF scoreF ValueH0
Sum of Squares Within220.812.018.4
Sum of Squares Between59.22.029.6
Sum of Squares Total280.014.0
Result

F score and F value

We can calculate the f score by dividing the mean square of between by the mean square of within. The square value will be in the scope of f distribution. After finding f value via the degree of freedom. We can determine whether the null hypothesis is rejected or accepted.

image 22
image 23
Python
Python
Python
# F score 
final_table['F score']['Result']=final_table['Mean Square']['Sum of Squares Between']/final_table['Mean Square']['Sum of Squares Within']

# F value
numerator=final_table['Degree of Freedom']['Sum of Squares Between']
denominator=final_table['Degree of Freedom']['Sum of Squares Within']
alpha=0.05
final_table['F Value']['Result']=scipy.stats.f.isf(alpha, numerator, denominator)

# Reject or fail to Reject Null hypothesis
final_table['H0']['Result']=final_table['F score']['Result']<final_table['F Value']['Result']
final_table
Sum of SquaresDegree of FreedomMean SquareF scoreF ValueH0
Sum of Squares Within220.812.018.4
Sum of Squares Between59.22.029.6
Sum of Squares Total280.014.0
Result1.6086963.885294True

So the null hypothesis is accepted in this instance. Try to run this program on your system. Your result may vary because our X is produced randomly.

Make a note that terms regarding null hypotheses are “reject the null hypothesis” or “fail to reject the null hypothesis.” But for ease of understanding, we used rejection or acceptance.

IQR

In this article, we will learn different methods to calculate IQR by hand.

Read More

IQR with Excel and Python

This article will teach us different methods to calculate IQR with Python and Excel.

Read More

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

  • ANCOVA: Analysis of Covariance with python

    ANCOVA is an extension of ANOVA (Analysis of Variance) that combines blocks of regression analysis and ANOVA. Which makes it Analysis of Covariance.

  • Learn Python The Fun Way

    What if we learn topics in a desirable way!! What if we learn to write Python codes from gamers data !!

  • Meet the most efficient and intelligent AI assistant : NotebookLM

    Start using NotebookLM today and embark on a smarter, more efficient learning journey!

  • Break the ice

    This can be a super guide for you to start and excel in your data science career.

  • Hypothesis Testing: A Comprehensive Overview

    This article delves into the application of hypothesis testing across diverse domains

  • Versions of ANCOVA (Analysis Of Covariance) with python

    To perform ANCOVA (Analysis of Covariance) with a dataset that includes multiple types of variables, you’ll need to ensure your dependent variable is continuous, and you can include categorical variables as factors. Below is an example using the statsmodels library in Python: Mock Dataset Let’s create a dataset with a mix of variable types: Performing…

  • Python Variables

    How useful was this post? Click on a star to rate it! Submit Rating Average rating 0 / 5. Vote count: 0 No votes so far! Be the first to rate this post.

  • A/B Testing Quiz

    Complete the code by dragging and dropping the correct functions

  • Python Functions

    Python functions are a vital concept in programming which enables you to group and define a collection of instructions. This makes your code more organized, modular, and easier to understand and maintain. Defining a Function: In Python, you can define a function via the def keyword, followed by the function name, any parameters wrapped in parentheses,…

  • Python Indexing: A Guide for Data Science Beginners

    Mastering indexing will significantly boost your data manipulation and analysis skills, a crucial step in your data science journey.

  • Diffusion Models: Making AI Creativity

    Stable Diffusion Models: Where Art and AI Collide Artificial Intelligence meets creativity in the fascinating realm of Stable Diffusion Models. These innovative models take text descriptions and bring them to life in the form of detailed and realistic images. Let’s embark on a journey to understand the magic behind Stable Diffusion in a way that’s…

  • Quiz Challenge: Basics with Python [Questions]

    Solve These Questions in Following Challange

  • Introducing Plethora of Stable Diffusion models: Part 1

    Generate AI images as good as DALL-E completely offline.

One response to “ANOVA (Analysis of Variance ) part 1”

  1. ANOVA (Analysis of Variance ) Part-2 - Distinctive Analytics

    […] only need to understand two or three concepts if you have read the ANOVA Part-1 article. We use two factors instead of one in a two-way ANOVA. What this means is that there will […]

Leave a Reply

Points You Earned

Untitled design 6
0 distinction_points
Untitled design 5
python_points 0
0 Solver points
Facebook
Instagram
WhatsApp
error: Content is protected !!