# UNIT 4: DATA COLLECTION

# 1. INTRODUCTION

In the modern digital landscape, data is the fundamental currency of progress. Every time you like a photo on social media, swipe a credit card, or search for a location on a map, you are generating data. For a computer scientist, learning how to collect, store, and process this information is the bridge between writing simple scripts and building powerful, real-world applications.

This unit, Data Collection, explores the technical and ethical dimensions of handling information in Java. We begin with the critical responsibility of the programmer: ensuring that data is used ethically and that algorithms remain unbiased. From there, we dive into the mechanisms of storage—moving from the static efficiency of Arrays to the dynamic flexibility of ArrayLists. We will learn how to pull information from external Text Files and organize complex information into 2D Arrays (matrices). Finally, we will master the logic of Searching and Sorting—the algorithms that allow us to find a needle in a digital haystack. By the end of this chapter, you will transition from a coder who manages variables to a developer who manages systems of information.

# 2. ALL KEY CONCEPTS, TERMS, FOUNDATIONAL KNOWLEDGE, AND PRINCIPLES

# Ethical and Social Foundations

• Personal Privacy: The right of an individual to control how their personal information is collected and used.
• Algorithmic Bias: Systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others.
• Data Integrity: The accuracy, completeness, and reliability of data throughout its lifecycle.

# Data Structures

• Array: A container object that holds a fixed number of values of a single type.
• Index: An integer value representing the position of an element in a collection. In Java, indices are zero-based ($0, 1, 2, \dots, n-1$).
• ArrayList: A class in the java.util package that provides a resizable-array implementation.
• 2D Array: An "array of arrays," structured like a table with rows and columns.
• Wrapper Class: A class whose object wraps or contains primitive data types (e.g., Integer for int).
• Autoboxing/Unboxing: The automatic conversion the Java compiler makes between primitive types and their corresponding object wrapper classes.

# Algorithms and Logic

• Traversal: The process of visiting every element in a data structure exactly once.
• Linear Search: A search algorithm that checks every element in a list sequentially until the target is found.
• Binary Search: An efficient search algorithm for sorted lists that repeatedly divides the search interval in half.
• Selection Sort: A sorting algorithm that repeatedly finds the minimum element from the unsorted part and puts it at the beginning.
• Insertion Sort: A sorting algorithm that builds the final sorted array one item at a time by "inserting" elements into their correct position.
• Merge Sort: A recursive "divide and conquer" sorting algorithm that splits an array in half, sorts the halves, and merges them.
• Recursion: A programming technique where a method calls itself to solve a smaller instance of the same problem.

# 3. IN-DEPTH EXPLANATION OF EVERY CONCEPT AND PRINCIPLE

# 4.1 – Ethical and Social Issues Around Data Collection

Programmers act as gatekeepers of information. When we collect data, we must consider:

1. Privacy: Are we collecting more than we need? Is the data encrypted?
2. Bias: If a dataset used to train an AI only includes data from one demographic, the resulting program will likely fail or perform poorly for other groups. This is algorithmic bias.
3. Utility: Data must be fit for its purpose. Using a dataset of "average rainfall" to predict "stock market trends" is a failure of data logic.

# 4.2 & 4.3 – Array Creation and Access

An array is a linear data structure. Once created, its length is immutable (cannot change).

• Declaration: int[] scores;
• Creation: scores = new int[5]; (Initializes elements to default: $0$ for numbers, false for boolean, null for objects).
• Initializer List: int[] primes = {2, 3, 5, 7};
• Access: Elements are accessed using arr[index].
• Boundary: For an array of length $L$, the valid indices are the set ${i \in \mathbb{Z} \mid 0 \le i < L}$. Accessing $L$ or higher triggers an ArrayIndexOutOfBoundsException.

# 4.4 & 4.5 – Array Traversals and Algorithms

We use loops to process arrays.

• Standard For Loop: for(int i = 0; i < arr.length; i++) — Used when we need the index.
• Enhanced For Loop (For-Each): for(int val : arr) — Used for reading. Warning: You cannot modify the original primitive values in the array using this loop variable because val is a copy.
• Standard Algorithms:
  • Sum/Average: $\bar{x} = \frac{\sum_{i=0}^{n-1} a_i}{n}$
  • Min/Max: Initialize a "current best" with the first element, then compare.

# 4.6 – Using Text Files

Data persistence allows information to survive after the program closes.

• Setup: Use import java.util.Scanner; and import java.io.File;.
• Scanner: Scanner fileReader = new Scanner(new File("data.txt"));
• Safety: Methods using Files must declare throws IOException.
• Logic: Always use while(fileReader.hasNext()) to prevent reading past the end of the file.

# 4.7 – Wrapper Classes

Primitives (int, double, boolean) are not objects. To store them in an ArrayList, Java uses Wrapper classes.

• Autoboxing: Double d = 5.5; (Converts primitive 5.5 to Double object).
• Parsing: int x = Integer.parseInt("123"); converts a String to an int.

# 4.8, 4.9, & 4.10 – ArrayLists

The ArrayList<E> is dynamic.

• Methods:
  • size(): Returns number of elements.
  • add(obj): Appends to the end.
  • add(index, obj): Inserts at index, shifts everything else right.
  • set(index, obj): Replaces element at index.
  • remove(index): Deletes and shifts everything else left.
• Traversal Danger: If you remove elements while looping forward, the indices of remaining elements shift, causing you to skip the next element. Solution: Loop backwards or decrement the index variable.

# 4.11, 4.12, & 4.13 – 2D Arrays

A 2D array is a grid. int[][] table = new int[rows][cols];.

• table.length = Number of rows.
• table[0].length = Number of columns.
• Row-Major: Outer loop traverses rows; inner loop traverses columns.
• Column-Major: Outer loop traverses columns; inner loop traverses rows.

# 4.14 & 4.15 – Searching and Sorting

• Linear Search: Average case visits $\frac{n}{2}$ elements.
• Selection Sort: $O(n^2)$. Find the smallest in the unsorted section, swap it to the front.
• Insertion Sort: $O(n^2)$. Iterate through, "sliding" each element back until it hits a smaller value.

# 4.16 & 4.17 – Recursion and Advanced Algorithms

Recursion requires a Base Case (the stopping condition) and a Recursive Call (the part where the function calls itself with a simpler version of the problem).

• Binary Search: Requires a sorted list. It checks the middle. If the target is smaller, it repeats on the left half. Complexity is $O(\log n)$.
• Merge Sort: Recursively divides the array until size is 1, then merges them in order. Complexity is $O(n \log n)$.

# 4. EXAMPLES

# Example 1: Algorithmic Bias Scenario

A company uses an algorithm to screen resumes. The training data only consists of past employees, most of whom were male engineers.

• Problem: The algorithm learns that "male" is a characteristic of success, creating a bias against female candidates.
• Solution: Programmers must audit the dataset for diversity before training.

# Example 2: Array Default Initialization

double[] measurements = new double[3];
// measurements[0] is 0.0
// measurements[1] is 0.0
// measurements[2] is 0.0

# Example 3: Finding the Minimum in an Array

To find the minimum value in array $A$:

1. Let $min = A[0]$.
2. For $i = 1$ to $n-1$:
3. If $A[i] < min$, then $min = A[i]$.
   $$min = \min(A_0, A_1, \dots, A_{n-1})$$

# Example 4: Calculating Average

Given an array int[] arr = {10, 20, 30};
$$\text{Sum} = 10 + 20 + 30 = 60$$
$$\text{Average} = \frac{60}{arr.length} = \frac{60}{3} = 20.0$$

# Example 5: ArrayIndexOutOfBoundsException

int[] data = {1, 2, 3};
System.out.println(data[3]); // ERROR: Indices are 0, 1, 2.

# Example 6: Enhanced For Loop Limitation

int[] nums = {1, 2, 3};
for (int n : nums) {
    n = n * 2; // n is a local copy
}
// nums is still {1, 2, 3}

# Example 7: Modifying Objects in Enhanced For Loop

Rectangle[] rects = {new Rectangle(5, 5)};
for (Rectangle r : rects) {
    r.setWidth(10); // This DOES change the object in the array
}

# Example 8: File Reading with Scanner

Scanner sc = new Scanner(new File("input.txt"));
while(sc.hasNext()) {
    String word = sc.next();
    System.out.println(word);
}
sc.close();

# Example 9: String.split() usage

String line = "Apple,Banana,Cherry";
String[] fruits = line.split(",");
// fruits[0] is "Apple", fruits[1] is "Banana"...

# Example 10: Autoboxing and Unboxing

ArrayList<Integer> list = new ArrayList<>();
list.add(5); // Autoboxing: int 5 -> Integer object
int first = list.get(0); // Unboxing: Integer -> int

# Example 11: ArrayList remove() during Traversal (The WRONG way)

// Removing all 2s
ArrayList<Integer> list = new ArrayList<>(Arrays.asList(1, 2, 2, 3));
for(int i = 0; i < list.size(); i++) {
    if(list.get(i) == 2) list.remove(i);
}
// Result: [1, 2, 3] because index shifted!

# Example 12: ArrayList remove() during Traversal (The RIGHT way)

for(int i = list.size() - 1; i >= 0; i--) {
    if(list.get(i) == 2) list.remove(i);
}
// Result: [1, 3]

# Example 13: 2D Array Declaration

int[][] matrix = { {1, 2}, {3, 4}, {5, 6} };
// matrix.length is 3 (rows)
// matrix[0].length is 2 (columns)

# Example 14: Row-Major Traversal Sum

int total = 0;
for(int r = 0; r < matrix.length; r++) {
    for(int c = 0; c < matrix[r].length; c++) {
        total += matrix[r][c];
    }
}

# Example 15: Column-Major Traversal

for(int c = 0; c < matrix[0].length; c++) {
    for(int r = 0; r < matrix.length; r++) {
        System.out.print(matrix[r][c] + " ");
    }
}

# Example 16: Linear Search Implementation

public int findTarget(int[] arr, int target) {
    for(int i = 0; i < arr.length; i++) {
        if(arr[i] == target) return i;
    }
    return -1;
}

# Example 17: Selection Sort Trace

Array: [5, 3, 8, 2]

1. Find min in [5, 3, 8, 2]: 2. Swap with 5. Result: [2, 3, 8, 5]
2. Find min in [3, 8, 5]: 3. Already in place. Result: [2, 3, 8, 5]
3. Find min in [8, 5]: 5. Swap with 8. Result: [2, 3, 5, 8]

# Example 18: Insertion Sort Logic

Array: [4, 3, 2]

1. 3 is less than 4. Shift 4 right, insert 3: [3, 4, 2]
2. 2 is less than 4 and 3. Shift 4, shift 3, insert 2: [2, 3, 4]

# Example 19: Recursive Factorial Trace ($n!$)

fact(3):

• Calls 3 * fact(2)
• fact(2) calls 2 * fact(1)
• fact(1) returns 1 (Base case)
• Trace back: $2 \times 1 = 2 \rightarrow 3 \times 2 = 6$.

# Example 20: Binary Search Calculation

Target: 7. Array: [1, 3, 5, 7, 9, 11, 13]

1. Low=$0$, High=$6$, Mid=$3$.
2. arr[3] is 7. Found!
3. Steps: $\log_2(7) \approx 3$.

# Example 21: Shift Elements Left

int first = arr[0];
for(int i = 0; i < arr.length - 1; i++) {
    arr[i] = arr[i+1];
}
arr[arr.length - 1] = first;

# Example 22: Reversing an Array

for(int i = 0; i < arr.length / 2; i++) {
    int temp = arr[i];
    arr[i] = arr[arr.length - 1 - i];
    arr[arr.length - 1 - i] = temp;
}

# Example 23: Counting Elements with Property

int count = 0;
for(int val : arr) {
    if(val % 2 == 0) count++;
}

# Example 24: Checking if ALL elements match property

boolean allEven = true;
for(int val : arr) {
    if(val % 2 != 0) {
        allEven = false;
        break;
    }
}

# Example 25: Identifying Duplicates

boolean hasDuplicate = false;
for(int i = 0; i < arr.length; i++) {
    for(int j = i + 1; j < arr.length; j++) {
        if(arr[i] == arr[j]) hasDuplicate = true;
    }
}

# Example 26: Accessing Consecutive Pairs

for(int i = 0; i < arr.length - 1; i++) {
    int diff = Math.abs(arr[i] - arr[i+1]);
}

# Example 27: Parsing a File of Numbers

Scanner sc = new Scanner(new File("nums.txt"));
double sum = 0;
int count = 0;
while(sc.hasNextDouble()) {
    sum += sc.nextDouble();
    count++;
}
double avg = sum / count;

# Example 28: 2D Array Column Sum

int colIndex = 1;
int sum = 0;
for(int r = 0; r < matrix.length; r++) {
    sum += matrix[r][colIndex];
}

# Example 29: Wrapper Static Methods

String val = "45.6";
double d = Double.parseDouble(val);
System.out.println(Integer.MAX_VALUE); // 2147483647

# Example 30: ArrayList set() vs add()

ArrayList<String> names = new ArrayList<>();
names.add("Alex"); // size 1
names.set(0, "Ben"); // size still 1, "Alex" is gone.

# Example 31: Merge Sort Visual Step

[38, 27, 43, 3]
Split: [38, 27] and [43, 3]
Split: [38], [27], [43], [3]
Merge: [27, 38], [3, 43]
Merge: [3, 27, 38, 43]

# Example 32: Binary Search (Recursive logic)

To find target $x$ in sorted $A$:
$f(low, high)$:

1. If $low > high$, return $-1$.
2. $mid = \frac{low+high}{2}$
3. If $A[mid] == x$, return $mid$.
4. If $A[mid] > x$, return $f(low, mid-1)$.
5. Else return $f(mid+1, high)$.

# Example 33: Simultaneous Traversal

Comparing two arrays of same length:

for(int i = 0; i < arr1.length; i++) {
    if(arr1[i] != arr2[i]) return false;
}

# Example 34: 2D Array "Flattening" logic

A $3 \times 3$ 2D array can be viewed as a 1D array of length $9$.
Element at arr[r][c] corresponds to index i = r * numCols + c.

# Example 35: Ethics of Data Incompleteness

A medical diagnostic tool is trained on data from adults only.

• Outcome: If used on children, the program may malfunction or provide dangerous dosage recommendations.
• Takeaway: Always evaluate if the dataset is appropriate for the target demographic.