Distinctive Analytics

A Place to enhance your analytical skills

ANOVA (Analysis of Variance ) part 1

Distinctive Analytics > Blog > Lessons > ANOVA (Analysis of Variance ) part 1

A method to find a statistical relationship between two variables in a dataset where one variable is used to group data.

Topics: , ,

Definition of ANOVA: A statistical method in which the variation in a set of observations is divided into distinct components. According to Oxford languages.

Simply put, it would be a method to find a statistical relationship between two variables in a dataset where one variable is used to group data.

The method to find the statistical significance is to calculate the variance across the whole dataset, the variance between groups, and the variance within groups.

To perform ANOVA on a dataset, we need one categorical and one continuous variable.

Types of ANOVA

  • One Way ANOVA
  • Two Way ANOVA

One Way ANOVA

One-way ANOVA is used to compare three or more groups based on one categorical and one continuous variable.

Import Required Libraries

Python
Python
Python
import pandas as pd
import numpy as np
import random
# Library to extract f-value
import scipy.stats

Create Dataset

Python
Python
Python
# create random x variable
x=random.sample(range(0, 16), 15)
data={'x':x,'y':[1,1,1,1,1,2,2,2,2,2,3,3,3,3,3]}
df=pd.DataFrame(data)
01234567891011121314151617
x234342202221202122353637363735
y111111222222333333
zaaabbbaaabbbaaabbb

Let’s start with grouping data. We will group X according to Y.

Python
Python
Python
for i in df['y'].unique():
    print(f'Group {i}')
    print(df[df['y']==i],end='\n\n')
Output
Group 1
   x  y  z
0  2  1  a
1  3  1  a
2  4  1  a
3  3  1  b
4  4  1  b
5  2  1  b

Group 2
     x  y  z
6   20  2  a
7   22  2  a
8   21  2  a
9   20  2  b
10  21  2  b
11  22  2  b

Group 3
     x  y  z
12  35  3  a
13  36  3  a
14  37  3  a
15  36  3  b
16  37  3  b
17  35  3  b
Python

Calculating the means of each group

Python
Python
Python
groupM=df.groupby('y').mean().reset_index()
groupM
Output
  y	 x
0	1	3.0
1	2	21.0
2	3	36.0
Python

Calculate the mean of all data (grand mean)

Python
Python
Python
grandM=df['x'].sum()/15
grandM
Output
20.0
Python

Calculate the Sum of Squares Total

image 13
Python
Python
Python
df['sst']=(df['x']-(df.x.sum()/len(df)))**2
SST=df['sst'].sum()
print(SST)
Output
3288.0
Python

Calculate the Sum of Squares Within

image 16
Python
Python
Python
SSW=0
for i in list(df['y'].unique()):
    #print(i)
    g=pd.DataFrame(df[df['y']==i].x-float(groupM[groupM['y']==i].x))**2
    SSW+=g.sum()
print(float(SSW))
Output
8490.0
Python

Calculate Sum of Squares Between

Python
Python
Python
groupM['ssb']=groupM.x-grandM
groupM['ssb']=groupM['ssb']**2 
groupM['ssb']=groupM['ssb']*N
SSB=groupM.ssb.sum()
print(SSB)
Output
2730.0
Python

All the knowledge that has been obtained up to this point is used to comprehend the occasion, setting, and context as well as the meaning of the statement.

Calculate the Degree of freedom for SST, SSW, SSB

The concept of degree of freedom is a method for comprehending logically independent values. By calculating the degree of freedom, we can get the scope of an interpretable sample of factors in the dataset.

Python
Python
Python
N=len((df[df['y']==1]))
M=len(df['y'].unique())
print(M,N)
Output
3 6
Python
Python
Python
Python
# Degrees of Freedom for all 
sstdf= (M*N)-1
sswdf= M*(N-1)
ssbdf= M-1

print(f'DF\n SST= {sstdf}\n SSW= {sswdf}\n SSB= {ssbdf}')
Output
DF
 SST= 17
 SSW= 15
 SSB= 2
Python

Values Table

Let’s create a table with all the values we have found so far.

Python
Python
Python
final_table=pd.DataFrame({
    'Sum of Squares':[float(SSW),float(SSB),float(SST),np.nan],
    'Degree of Freedom':[sswdf,ssbdf,sstdf,np.nan],
    'Mean Square':[np.nan for x in range(4)],
    'F score':[np.nan for x in range(4)],
    'F Value':[np.nan for x in range(4)],
    'H0':[np.nan for x in range(4)]} ,
    index=['Sum of Squares Within','Sum of Squares Between','Sum of Squares Total','Result'])
final_table
Sum of SquaresDegree of FreedomMean SquareF scoreF ValueH0
Sum of Squares Within220.812.0
Sum of Squares Between59.22.0
Sum of Squares Total280.014.0
Result

Mean Square

To calculate the mean square, we divide the degree of freedom by the respective sum of squares.

image 20
Python
Python
Python
final_table.loc[:'Sum of Squares Between']['Mean Square']=final_table.loc[:'Sum of Squares Between']['Sum of Squares']/final_table.loc[:'Sum of Squares Between']['Degree of Freedom']
final_table
Sum of SquaresDegree of FreedomMean SquareF scoreF ValueH0
Sum of Squares Within220.812.018.4
Sum of Squares Between59.22.029.6
Sum of Squares Total280.014.0
Result

F score and F value

We can calculate the f score by dividing the mean square of between by the mean square of within. The square value will be in the scope of f distribution. After finding f value via the degree of freedom. We can determine whether the null hypothesis is rejected or accepted.

image 22
image 23
Python
Python
Python
# F score 
final_table['F score']['Result']=final_table['Mean Square']['Sum of Squares Between']/final_table['Mean Square']['Sum of Squares Within']

# F value
numerator=final_table['Degree of Freedom']['Sum of Squares Between']
denominator=final_table['Degree of Freedom']['Sum of Squares Within']
alpha=0.05
final_table['F Value']['Result']=scipy.stats.f.isf(alpha, numerator, denominator)

# Reject or fail to Reject Null hypothesis
final_table['H0']['Result']=final_table['F score']['Result']<final_table['F Value']['Result']
final_table
Sum of SquaresDegree of FreedomMean SquareF scoreF ValueH0
Sum of Squares Within220.812.018.4
Sum of Squares Between59.22.029.6
Sum of Squares Total280.014.0
Result1.6086963.885294True

So the null hypothesis is accepted in this instance. Try to run this program on your system. Your result may vary because our X is produced randomly.

Make a note that terms regarding null hypotheses are “reject the null hypothesis” or “fail to reject the null hypothesis.” But for ease of understanding, we used rejection or acceptance.

IQR

In this article, we will learn different methods to calculate IQR by hand.

Read More

IQR with Excel and Python

This article will teach us different methods to calculate IQR with Python and Excel.

Read More

How useful was this post?

Click on a star to rate it!

  • ANCOVA: Analysis of Covariance with python

    ANCOVA is an extension of ANOVA (Analysis of Variance) that combines blocks of regression analysis and ANOVA. Which makes it Analysis of Covariance.

  • Learn Python The Fun Way

    What if we learn topics in a desirable way!! What if we learn to write Python codes from gamers data !!

  • Meet the most efficient and intelligent AI assistant : NotebookLM

    Start using NotebookLM today and embark on a smarter, more efficient learning journey!

  • Break the ice

    This can be a super guide for you to start and excel in your data science career.

  • Tourism Trend Prediction

    After tourism was established as a motivator of local economies (country, state), many governments stepped up to the plate.

  • Sentiment Analysis Polarity Detection using pos tag

    Sentiment analysis can determine the polarity of sentiments from given sentences. We can classify them into certain categories.

  • For loop with Dictionary

    Traverse a dictionary with for loop Accessing keys and values in dictionary. Use Dict.values() and Dict.keys() to generate keys and values as iterable. Nested Dictionaries with for loop Access Nested values of Nested Dictionaries How useful was this post? Click on a star to rate it! Submit Rating

  • For Loops with python

    For loop is one of the most useful methods to reuse a code for repetitive execution.

  • Metrics and terminologies of digital analytics

    These all metrics are revolving around visits and hits which we are getting on websites. Single page visits, Bounce, Cart Additions, Bounce Rate, Exit rate,

  • Hypothesis Testing

    Hypothesis testing is a statistical method for determining whether or not a given hypothesis is true. A hypothesis can be any assumption based on data.

  • A/B testing

    A/B tests are randomly controlled experiments. In A/B testing, you get user response on various versions of the product, and users are split within multiple versions of the product to figure out the “winner” of the version.

  • For Loop With Tuples

    This article covers ‘for’ loops and how they are used with tuples. Even if the tuples are immutable, the accessibility of the tuples is similar to that of the list.

  • Multivariate ANOVA (MANOVA) with python

    MANOVA is an update of ANOVA, where we use a minimum of two dependent variables.

  • Two-Way ANOVA

    You only need to understand two or three concepts if you have read the one-way ANOVA article. We use two factors instead of one in a two-way ANOVA.

One response to “ANOVA (Analysis of Variance ) part 1”

  1. ANOVA (Analysis of Variance ) Part-2 - Distinctive Analytics

    […] only need to understand two or three concepts if you have read the ANOVA Part-1 article. We use two factors instead of one in a two-way ANOVA. What this means is that there will […]

Leave a Reply

Points You Earned

Untitled design 6
0 distinction_points
Untitled design 5
python_points 0
0 Solver points
Facebook
Instagram
WhatsApp
error: Content is protected !!