Total Pageviews

Thursday, August 13, 2026

Pandas for data science

 

  • Handles missing data (NaN, NA, NaT easily).
  • Flexible size mutability (insert/delete columns).
  • Automatic data alignment (aligns labels automatically).
  • Powerful groupby engine (split-apply-combine operations).
  • Easy data conversion (converts structures to DataFrames).
  • Smart indexing (label-based slicing and subsetting).
  • Intuitive data merging (easy joining of datasets).
  • Flexible reshaping (pivoting and restructuring data).
  • Hierarchical axis labeling (multiple labels per tick).
  • Robust I/O tools (loads CSV, Excel, HDF5).
  • Time-series ready (date shifting and frequency conversion).

  • Series: One-dimensional array-like structure.
  • DataFrame: Two-dimensional, size-mutable tabular structure.

1.
import pandas as pd
s = pd.Series([1, 3, 5, 7, 9])
print(s)


0    1
1    3
2    5
3    7
4    9
dtype: int64




2.

data = {'Name': ['John', 'Anna', 'Peter', 'Linda'],
        'Age': [28, 34, 29, 32]}
df = pd.DataFrame(data)
print(df)


    Name  Age
0   John   28
1   Anna   34
2  Peter   29
3  Linda   32



3.

# Creating a DataFrame from a dictionary. import pandas as pd data = { 'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [25, 30, 35], 'City': ['New York', 'Paris', 'London'] } df = pd.DataFrame(data) print(df)



4. # To look at the first few rows of a DataFrame. print(df.head()) # First five rows

      Name  Age      City
0    Alice   25  New York
1      Bob   30     Paris
2  Charlie   35    London



5.
# To access a column in the DataFrame.
ages = df['Age']
print(ages)
print(df['Name']) # Displays the 'Name' column

0    25
1    30
2    35
Name: Age, dtype: int64
0      Alice
1        Bob
2    Charlie
Name: Name, dtype: object


6.

subset = df[0:2] # First two rows print(subset)


    Name  Age      City
0  Alice   25  New York
1    Bob   30     Paris


7.


filtered_df = df[df['Age'] > 30] # Rows where age is greater than 30 print(filtered_df)

      Name  Age    City
2  Charlie   35  London



8.

df['AgeInTenYears'] = df['Age'] + 10 print(df)

      Name  Age      City  AgeInTenYears
0    Alice   25  New York             35
1      Bob   30     Paris             40
2  Charlie   35    London             45



9.
print(df.describe())

        Age  AgeInTenYears
count   3.0            3.0
mean   30.0           40.0
std     5.0            5.0
min    25.0           35.0
25%    27.5           37.5
50%    30.0           40.0
75%    32.5           42.5
max    35.0           45.0


10.
f.rename(columns={'AgeYears': 'Age'}, inplace=True) #print(df['Age'].mean()) # Average age


11.
df print(df['Age'].mean()) # Average age

30.0


12.
df.plot(kind='line')





df['Age'].plot(kind='bar')



# Select rows where 'Age' is greater than 30 print(df[df['Age'] > 30]) filtered_df = df[df['Age'] > 30] # Rows where age is greater than 30 print(filtered_df)

     Name   Age    City  AgeInTenYears
2  Charlie  35.0  London             45
      Name   Age    City  AgeInTenYears
2  Charlie  35.0  London             45















#DATA CLEANING #Drop Rows With Missing Values import pandas as pd # Define a dictionary with Indian employee data containing missing values (None) data = { "Employee": ["Aarav", "Priya", "Rahul", "Ananya", "Kabir"], "Salary_LPA": [12.0, None, 18.5, 9.0, 15.0], # Priya's salary is missing "City": ["Mumbai", "Delhi", None, "Bangalore", None], # Rahul and Kabir have missing cities } df = pd.DataFrame(data) print("Original Data:\n", df) print() # Use dropna() to remove rows that contain any missing values df_cleaned = df.dropna() print("Cleaned Data (Rows with any missing values removed):\n", df_cleaned)


Original Data:
   Employee  Salary_LPA       City
0    Aarav        12.0     Mumbai
1    Priya         NaN      Delhi
2    Rahul        18.5       None
3   Ananya         9.0  Bangalore
4    Kabir        15.0       None

Cleaned Data (Rows with any missing values removed):
   Employee  Salary_LPA       City
0    Aarav        12.0     Mumbai
3   Ananya         9.0  Bangalore




#Fill Missing Values import pandas as pd # define a dictionary with sample data which includes some missing values data = { 'A': [1, 2, 3, None, 5], 'B': [None, 2, 3, 4, 5], 'C': [1, 2, None, None, 5] } df = pd.DataFrame(data) print("Original Data:\n", df) # filling NaN values with 0 df.fillna(0, inplace=True) print("\nData after filling NaN with 0:\n", df)



Original Data:
      A    B    C
0  1.0  NaN  1.0
1  2.0  2.0  2.0
2  3.0  3.0  NaN
3  NaN  4.0  NaN
4  5.0  5.0  5.0

Data after filling NaN with 0:
      A    B    C
0  1.0  0.0  1.0
1  2.0  2.0  2.0
2  3.0  3.0  0.0
3  0.0  4.0  0.0
4  5.0  5.0  5.0




import pandas as pd # define a dictionary with sample data which includes some missing values data = { 'A': [1, 2, 3, None, 5], 'B': [None, 2, 3, 4, 5], 'C': [1, 2, None, None, 5] } df = pd.DataFrame(data) print("Original Data:\n", df) # filling NaN values with the mean of each column df.fillna(df.mean(), inplace=True) print("\nData after filling NaN with mean:\n", df)



Original Data:
      A    B    C
0  1.0  NaN  1.0
1  2.0  2.0  2.0
2  3.0  3.0  NaN
3  NaN  4.0  NaN
4  5.0  5.0  5.0

Data after filling NaN with mean:
       A    B         C
0  1.00  3.5  1.000000
1  2.00  2.0  2.000000
2  3.00  3.0  2.666667
3  2.75  4.0  2.666667
4  5.00  5.0  5.000000



import pandas as pd # sample data data = { 'A': [1, 2, 2, 3, 3, 4], 'B': [5, 6, 6, 7, 8, 8] } df = pd.DataFrame(data) print("Original DataFrame:\n", df.to_string(index=False)) # detect duplicates print("\nDuplicate Rows:\n", df[df.duplicated()].to_string(index=False)) # remove duplicates based on column 'A' df.drop_duplicates(subset=['A'], keep='first', inplace=True) print("\nDataFrame after removing duplicates based on column 'A':\n", df.to_string(index=False))

Original DataFrame: A B 1 5 2 6 2 6 3 7 3 8 4 8 Duplicate Rows: A B 2 6 DataFrame after removing duplicates based on column 'A': A B 1 5 2 6 3 7 4 8















matplotlib

SciPy

 SciPy

NumPy

 NumPy

scikit-learn

 Scikit-Learn provides core machine learning tools. These include model evaluation functions like accuracy_score, data preparation tools like StandardScaler, and model selection helpers like train_test_split. It also features clustering algorithms like KMeans, classification models like SVC, and regression models like LinearRegression.

Evaluation and Preprocessing
  • Measures accuracy and error with accuracy_score and mean_squared_error
  • Builds confusion matrices with confusion_matrix
  • Scales data using StandardScaler and MinMaxScaler
  • Encodes labels with LabelEncoder
  • Fills missing data via SimpleImputer 
Selection and Clustering
  • Splits data using train_test_split
  • Tunes parameters with GridSearchCV
  • Validates models using cross_val_score
  • Groups data with KMeans
  • Uses hierarchical groups via AgglomerativeClustering 
Classification and Regression
  • Predicts classes with KNeighborsClassifier and SVC
  • Uses trees and Bayes via DecisionTreeClassifier and GaussianNB
  • Fits linear equations with LinearRegression and Lasso
  • Predicts values using RandomForestRegressor 

4. image input and display (gray scale)

 


8. Image Thresholding using OpenCV (or Image Binarization).








 

7. Image Thresholding using OpenCV (or Image Binarization).








 

6. OpenCV image reading and writing

 


5. OpenCV image reading and writing

 5. 





3. Get number of pixel, dimension of image

 import cv2


img = cv2.imread('im1.jpg')


cv2.imshow("Cute Kitens", img)

print("Image Properties")

print("- Number of Pixels: " + str(img.size))

print("- Shape/Dimensions: " + str(img.shape))



OUTPUT:

Image Properties
- Number of Pixels: 489402
- Shape/Dimensions: (318, 513, 3)


2. image input and display (color)

 import cv2

img = cv2.imread("im.jpeg", cv2.IMREAD_COLOR)

cv2.imshow("Cute Kitens", img)

cv2.waitKey(0)

cv2.destroyAllWindows()

output:






1. image input and display



CODE :


 import cv2

img = cv2.imread("im.jpeg")

cv2.imshow("Cute Kitens", img)

cv2.waitKey(0)

cv2.destroyAllWindows()