Predicting Customer Churn with AI Agents

In this article, we'll use our AI data analyst to analyze and predict customer churn.

We'll use data from Kaggle on Telco Customer Churn and run it through the customer churn analysis workflow, which is an agentic workflow using OpenAI's Assistants API with the following prompts:

Using the churn prediction outputs from this agent, we've also put the findings into an interactive dashboard with both the churn predictions, analysis, and input data:

Executive Summary

This report presents a comprehensive analysis of customer churn using a dataset of 7,043 Telco customers from Kaggle to identify key churn predictors, develop predictive models, and formulate actionable insights.

Through data preparation, feature engineering, and model evaluation, significant predictors of churn were identified. Logistic regression was selected as the optimal model based on performance metrics, and targeted strategies were proposed to reduce churn.

Table of Contents

1. Introduction and Data Overview
2. Data Preparation and Feature Engineering
3. Churn Prediction Methodology
4. Model Performance and Visualizations
5. Key Insights and Recommendations
6. Prediction Output CSV
7. Appendix: Code Snippeters

1. Introduction and Data Overview

The dataset comprises customer information including demographics, contract details, payment methods, and usage metrics. It aims to predict customer churn, providing businesses insight into retention strategies. The primary focus was to identify predictors using machine learning models.

Let's start by examining the contents of the uploaded file to understand its structure, content, and the types of variables it contains. This will guide us in preparing the data for analysis, particularly for identifying potential predictors of customer churn.

Data Preparation Steps

The dataset contains 7,043 entries and 21 columns detailing customer information. The target variable for churn analysis is Churn. Here's how the data preparation will proceed:

1. Churn Predictor Identification: Selecting variables that can potentially predict churn.
2. Missing Value Treatment: Handling any missing values in the dataset.
3. Feature Encoding and Normalization: Encoding categorical variables and normalizing numerical features.
4. Data Preprocessing: Applying the relevant transformations as per the analysis requirements.

Churn Predictor Identification

From the exploratory look, the following variables are potential predictors of customer churn:

• gender
• SeniorCitizen
• Partner
• Dependents
• tenure
• PhoneService
• MultipleLines
• InternetService
• OnlineSecurity
• OnlineBackup
• DeviceProtection
• TechSupport
• StreamingTV
• StreamingMovies
• Contract
• PaperlessBilling
• PaymentMethod
• MonthlyCharges
• TotalCharges

Missing Value Treatment

Let's check if there are any missing or improperly formatted values in our numerical features like TotalCharges.

Feature Encoding and Normalization

• Categorical Encoding: Convert categorical variables such as gender, Partner, etc., to numerical representations using techniques like one-hot encoding or label encoding.
• Numerical Normalization: Scale numerical variables such as tenure, MonthlyCharges, and TotalCharges to a standardized range.

Preprocessing Steps

We will preprocess the data by:

• Handling missing values.
• Encoding categorical features.
• Normalizing numerical features.

Next, let's handle missing values and format any improperly formatted columns. Then process categorical encodings and normalize numerical feature.

2. Data Preparation and Feature Engineering

Summary

• Churn Predictor Identification: Variables like contract type, monthly charges, and billing method identified as potential predictors.
• Missing Values: Addressed missing TotalCharges values using median imputation.
• Feature Encoding: Categorical variables were encoded using one-hot encoding. Numerical features were normalized to optimize model performance.

Missing Value Treatment

• We found 11 missing values in the TotalCharges column after attempting to convert it to numeric, likely due to non-numeric entries.
• We'll handle these missing values, perhaps by imputing them with meaningful values like the median or simply removing these rows, depending on the rest of the data distribution.

Feature Encoding and Normalization

Categorical Variables:

• We'll convert categorical features such as gender, Partner, InternetService, etc., to numerical values using one-hot encoding.
• The Churn column, our target variable, will be also encoded from Yes/No to binary values.

Numerical Variables:

• Normalize the features tenure, MonthlyCharges, and TotalCharges to ensure they have a mean of 0 and a standard deviation of 1 for better model performance.

Next, I will proceed with filling the missing values, encoding categorical features, and normalizing numerical features. Let's apply these transformations.

Final Data Preparation Summary

Missing Value Treatment:

• Handled missing TotalCharges values by imputing them with the median. This approach is chosen to prevent biasing the data with extreme values that could occur with mean imputation.

Feature Encoding:

• Applied one-hot encoding to transform categorical variables into numerical format, resulting in new features representing categorical variable states.
• The target variable Churn was transformed into a binary variable with values 0 (No) and 1 (Yes).

Feature Normalization:

• Normalized tenure, MonthlyCharges, and TotalCharges using standard scaling to ensure they have a mean of 0 and a standard deviation of 1. This step is essential for many machine learning algorithms that are sensitive to the scale of input data.

The data is now preprocessed and ready for further analysis or modeling to identify and assess variables that predict customer churn.

3. Churn Prediction Methodology

We employed multiple models including Logistic Regression, Decision Tree, Random Forest, and SVM, comparing them on accuracy, precision, recall, and F1 score. Logistic Regression emerged as the most effective model, striking a balance between interpretability and accuracy.

To predict customer churn, we will use several machine learning models. The choice of models is guided by their diversity in approach, which will allow us to compare and understand which model performs best given the characteristics of the data.

Churn Prediction Models

Model Selection and Rationale

We will consider the following models:

1. Logistic Regression: A simple, interpretable classifier that works well for binary classification tasks, particularly useful for understanding feature importance.
2. Decision Tree Classifier: Provides insight into data structure with if-then rules, useful for exploring the relationships in the data.
3. Random Forest Classifier: An ensemble method that leverages multiple decision trees to build more robust and less overfit models.
4. Support Vector Machine (SVM): Effective in high-dimensional spaces and useful where decision boundaries are complex.

Model Training Process

• Split the data into training and testing sets to validate performance.
• Train each model on the training dataset.
• Use the testing dataset to evaluate model performance.

Performance Evaluation

We will evaluate the models using the following metrics:

• Accuracy: Overall correctness of the model.
• Precision: Correct positive predictions against all positive predictions.
• Recall (Sensitivity): Correct positive predictions against all actual positives.
• F1 Score: Harmonic mean of precision and recall.

These metrics will help in understanding the trade-offs between identifying true churn cases without too many false positives.

Model Comparison and Selection

We'll compare the models based on their performance metrics and select the best-performing model as per the imbalance or importance of precision versus recall.

Let's proceed to apply these methods and evaluate their performance.

Churn Prediction Models: Results and Evaluation

Model Performance Evaluation

The models produced the following results in terms of accuracy, precision, recall, and F1 score:

Model Comparison and Selection

• Logistic Regression seems to be the best performer overall, with the highest F1 score, meaning it balances precision and recall better than other models.
• Decision Tree has lower performance metrics, indicating less robustness in handling the data.
• Random Forest and SVM offer competitive performance, with Random Forest providing slightly better recall than SVM but lower precision.

In this context, Logistic Regression is selected due to its balance of simplicity, interpretability, and effectiveness reflected in its precision and F1 score. This might be preferable when understanding the contribution of individual variables to churn is crucial.

Further tuning of hyperparameters could potentially improve the model performance. 

4. Model Performance and Visualizations

Let's create the visualizations that illustrate the most significant predictors of churn and the performance of the logistic regression model, as well as highlight customers at the highest risk of churn.

Churn Analysis Visualizations

Feature Importance Plot

For logistic regression, feature importance can be inferred from the absolute values of the coefficients. Larger coefficients indicate features with more predictive power.

ROC Curve and AUC Score

The ROC curve will help visualize the model's ability to discriminate between the classes, with the AUC representing the degree of separability.

Confusion Matrix

This will show the number of true positive, false positive, true negative, and false negative predictions, providing insights into model effectiveness.

High-Risk Customer Segment Analysis

Identify the top customers with the highest predicted probability of churn and analyze their characteristics.

Churn Analysis Visualizations

Feature Importance Plot

The feature importance plot displays which variables have the most influence on the logistic regression model. Features like Contract, MonthlyCharges, and PaperlessBilling have shown significant importance in predicting churn.

ROC Curve and AUC Score

The ROC curve demonstrates the trade-off between sensitivity and specificity for our logistic regression model, with an AUC score measuring its ability to distinguish between churn and non-churn classes. The higher the AUC, the better the model's performance.

Confusion Matrix

The confusion matrix provides insight into the model's predictive accuracy, showing: - True Positives (Correctly predicted churn) - False Positives (Incorrectly predicted churn) - True Negatives (Correctly predicted non-churn) - False Negatives (Incorrectly predicted non-churn)

High-Risk Customer Segment Analysis

A segment of customers identified as high risk (with a probability above 0.7) was analyzed. Characteristics include: - Majority being users with high MonthlyCharges. - Greater tendency to have Electronic check as a payment method. - Often no dependents and single-line contracts.

These insights can be used to craft targeted customer retention strategies. 

Performance Metric Insights

• Logistic Regression achieved an accuracy of 80.79%, with significant precision and F1 score.
• ROC curve analysis yielded a favorable AUC, showcasing the model's robust discriminatory capability.
• The confusion matrix detailed high true positive rates with manageable false positives/negatives.

Data Visualization Insights

• Feature Importance: Highlighted critical factors like contract type and monthly charges.
• ROC Curve and AUC Score: Demonstrated effective model discrimination between churn and retention.
• Confusion Matrix: Provided clarity on model prediction accuracy.

5. Key Insights and Recommendations

Summary:

• Major Churn Predictors: Contract type, monthly charges, and billing method identified as significant churn indicators.
• High-Risk Profiles: High monthly charge customers on month-to-month contracts are notably at risk.
• Strategies for Reduction: Suggestions include offering bundled services, improving billing processes, and enhancing engagement initiatives.

Major Churn Predictors

From the analysis, important factors predicting customer churn include:

• Contract Type: Month-to-month contracts are strongly associated with higher churn compared to one-year or two-year contracts.
• Monthly Charges: Customers with higher monthly charges are more likely to churn.
• Paperless Billing: Customers using paperless billing seem to have a higher churn rate.
• Payment Method: Those who use electronic checks are more prone to churn compared to those using credit cards, bank transfers, or mailed checks.

High-Risk Customer Profiles

Analysis of high churn risk customers reveals common characteristics:

• Service Plan: Tend to have short-term, month-to-month plans.
• Billing: Often utilize paperless billing and electronic check payment methods.
• Service Utilization: Typically have no additional services like multiple phone lines or bundled internet options.
• Demographics: Includes both senior and non-senior citizens, but a higher proportion of them do not have dependents or partners listed.

Actionable Strategies for Churn Reduction

1. Contract Incentives: Encourage long-term contracts by offering discounts or bonuses for signing one-year or two-year plans.
2. Billing Optimization: Introduce billing options with clearer, more flexible billing processes or discounts for using automatic bank transfers instead of electronic checks.
3. Customized Service Bundles: Offer bundled services to promote the addition of internet, phone lines, or TV packages, enhancing perceived value and stickiness.
4. Value Communication: Clearly communicate the value benefits of additional services during key customer interactions or through targeted marketing.

Customer Retention Recommendations

1. Proactive Engagement: Use predictive data analytics to identify at-risk customers and proactively reach out with personalized offers or loyalty programs.
2. Feedback Loops: Implement regular feedback mechanisms to understand customer pain points and improve service quality.
3. Customer Support: Enhance customer support channels and train representatives to handle inquiries with empathy, potentially converting churn risk into loyalty.
4. Experience Enhancement: Continuously work on improving user interfaces and experience throughout billing and account management systems.

Implementing these strategies will help lower the churn rate by addressing specific patterns associated with high-risk profiles and capitalize on the identified pivotal factors.

6. Churn Prediction Output

The prediction CSV includes customer ID, actual and predicted churn, churn risk level, and tailored recommendations. It serves as a tool for targeted retention strategies.

You can download the CSV file containing the customer churn predictions using this link below:

File Structure

• CustomerID: Unique identifier for each customer.
• ActualChurn: Binary indicator of actual churn (1 for churn, 0 for not churn).
• PredictedProbability: Probability of churn as predicted by the model.
• PredictedChurn: Binary prediction of churn (1 for predicted churn, 0 for not predicted churn).
• ChurnRisk: Categorized risk level based on predicted probability (Low, Medium, High).
• Recommendations: Actionable recommendations for customer retention based on churn risk levels.

Usage Guidelines and Interpretation

• Reading the File: Use tools such as Excel or a CSV reader to load and analyze the data.
• Interpreting Predictions:
• High Predicted Probability: Indicates a high likelihood of the customer churning.
• Churn Risk Level: Categorizes customers to prioritize actions.
• Recommendations: Provides strategies tailored to the customer's risk category.

This output can drive strategic decisions by targeting interventions that are specific to each customer's risk profile, ultimately aiming to reduce churn and improve retention. 

7. Appendix: Code Snippets

This section includes in-depth details on data cleaning, feature selection, and model training processes, along with complete code snippets used for analysis.

Code Snippet 1

from sklearn.preprocessing import StandardScaler

# Impute missing TotalCharges with median
data['TotalCharges'].fillna(data['TotalCharges'].median(), inplace=True)

# Encode categorical variables
data_encoded = pd.get_dummies(data.drop(['customerID'], axis=1), drop_first=True)

# Normalize numerical features
scaler = StandardScaler()
numerical_features = ['tenure', 'MonthlyCharges', 'TotalCharges']

# Apply scaling and updating the data
data_encoded[numerical_features] = scaler.fit_transform(data_encoded[numerical_features])

data_encoded.head()

• Purpose: Prepares the dataset by handling missing values, encoding categorical variables, and normalizing numerical features.
• Relevance: Ensures the dataset is clean and suitable for machine learning algorithms, which generally require normalized numerical features and no missing data.

Code Snippet 2

• Purpose: Checks the dataset for missing values, especially in the TotalCharges column, and converts it to a numeric type.
• Relevance: Forms part of the data preparation process. Ensures the dataset is clean and properly formatted, which is necessary for accurate modeling. This precedes the descriptive insights about missing data.

# Check for missing values and proper data type formats
data.isnull().sum(), data.dtypes

# Convert TotalCharges to numeric and handle any non-numeric as NaN
data['TotalCharges'] = pd.to_numeric(data['TotalCharges'], errors='coerce')

# Check the data again for any missing values
missing_values = data.isnull().sum()
data.info(), missing_values

Code Snippet 3

• Purpose: Loads the dataset from a CSV file, utilizing pandas to read and display the first few rows and a summary for understanding initial data characteristics.

import pandas as pd

# Load the uploaded data file
file_path = '/mnt/data/file-wmfNm8RPbifvwH4DYbT2nUZt'
data = pd.read_csv(file_path)

# Display the first few rows and summary of the dataset
data.head(), data.info()

Code Snippet 4

• Purpose: Evaluates different machine learning models, excluding XGBoost for faster execution.
• Relevance: Streamlines the model evaluation process. Important for comparing the performance of various models to determine the best one for churn prediction.

# Remove XGBoost for quick evaluation of other models
models.pop('XGBoost', None)

# Train and evaluate remaining models
results = {}
for model_name, model in models.items():
    # Train the model
    model.fit(X_train, y_train)
    
    # Predict on the test set
    y_pred = model.predict(X_test)
    
    # Calculate performance metrics
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred)
    recall = recall_score(y_test, y_pred)
    f1 = f1_score(y_test, y_pred)
    
    # Store the results
    results[model_name] = {
        'Accuracy': accuracy,
        'Precision': precision,
        'Recall': recall,
        'F1 Score': f1
    }

results

Code Snippet 5

• Purpose: Splits the dataset into training and testing sets, initializes different machine learning models, trains them, and evaluates their performance.
• Relevance: Core of the churn prediction methodology. This snippet lays the groundwork for selecting and evaluating machine learning models for churn prediction, providing the performance metrics used in the analysis.

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

# Splitting the data into training and testing sets
X = data_encoded.drop('Churn_Yes', axis=1)
y = data_encoded['Churn_Yes']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)

# Models to evaluate
models = {
    'Logistic Regression': LogisticRegression(max_iter=1000),
    'Decision Tree': DecisionTreeClassifier(),
    'Random Forest': RandomForestClassifier(),
    'SVM': SVC(probability=True),
    'XGBoost': XGBClassifier(use_label_encoder=False, eval_metric='logloss')
}

# Train and evaluate models
results = {}
for model_name, model in models.items():
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred)
    recall = recall_score(y_test, y_pred)
    f1 = f1_score(y_test, y_pred)
    
    results[model_name] = {
        'Accuracy': accuracy,
        'Precision': precision,
        'Recall': recall,
        'F1 Score': f1
    }

results

Code Snippet 6

• Purpose: Generates visualizations like feature importance plot, ROC curve, and a confusion matrix for the logistic regression model, and performs high-risk customer analysis.
• Relevance: Provides a visual representation of model performance and model insights. These visualizations support the analysis in the report by showing how predictions align with observed outcomes.

import matplotlib.pyplot as plt
from sklearn.metrics import roc_curve, auc, confusion_matrix, ConfusionMatrixDisplay

# Logistic regression model for visualizing feature importance
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

# Feature Importance Plot
importance = abs(model.coef_[0])
feature_names = X.columns
sorted_indices = importance.argsort()[::-1]

plt.figure(figsize=(12, 6))
plt.bar(range(len(sorted_indices)), importance[sorted_indices], align='center')
plt.xticks(range(len(sorted_indices)), [feature_names[i] for i in sorted_indices], rotation=90)
plt.title('Feature Importance for Logistic Regression')
plt.xlabel('Features')
plt.ylabel('Importance')
plt.tight_layout()
plt.show()

# ROC Curve and AUC Score
y_pred_prob = model.predict_proba(X_test)[:, 1]
fpr, tpr, _ = roc_curve(y_test, y_pred_prob)
roc_auc = auc(fpr, tpr)

plt.figure(figsize=(8, 6))
plt.plot(fpr, tpr, color='darkorange', lw=2, label=f'ROC curve (area = {roc_auc:0.2f})')
plt.plot([0, 1], [0, 1], color='navy', linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc="lower right")
plt.show()

# Confusion Matrix
y_pred = model.predict(X_test)
cm = confusion_matrix(y_test, y_pred)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=model.classes_)
disp.plot(cmap=plt.cm.Blues)
plt.title('Confusion Matrix')
plt.show()

# High-Risk Customer Segment Analysis
y_train_pred_prob = model.predict_proba(X_train)[:, 1]
risk_threshold = 0.7
high_risk_indices = y_train_pred_prob > risk_threshold
high_risk_customers = X_train[high_risk_indices]

# Summary of high-risk customers
high_risk_summary = high_risk_customers.describe()
high_risk_summary

Code Snippet 7

• Purpose: Generates and exports churn predictions with insights and recommendations for each customer to a CSV file.
• Relevance: Delivers actionable outputs that can be used for customer engagement strategies. It encapsulates findings into a format that stakeholders can utilize for strategic decisions.

# Prepare churn predictions and insights
customer_predictions = pd.DataFrame()
customer_predictions['CustomerID'] = data['customerID']
customer_predictions['ActualChurn'] = data['Churn'].apply(lambda x: 1 if x == 'Yes' else 0)
customer_predictions['PredictedProbability'] = model.predict_proba(X)[:, 1]
customer_predictions['PredictedChurn'] = model.predict(X)
customer_predictions['ChurnRisk'] = pd.cut(customer_predictions['PredictedProbability'],
                                             bins=[0, 0.33, 0.66, 1],
                                             labels=['Low', 'Medium', 'High'])

# Recommendations based on churn risk
customer_predictions.loc[customer_predictions['ChurnRisk'] == 'High', 'Recommendations'] = 'Consider long-term contract incentives'
customer_predictions.loc[customer_predictions['ChurnRisk'] == 'Medium', 'Recommendations'] = 'Offer bundled services'
customer_predictions.loc[customer_predictions['ChurnRisk'] == 'Low', 'Recommendations'] = 'Monitor and maintain current satisfaction levels'

# Select relevant columns to export
output_df = customer_predictions[['CustomerID', 'ActualChurn', 'PredictedProbability', 'PredictedChurn', 'ChurnRisk', 'Recommendations']]

# Save to CSV file
output_df.to_csv('/mnt/data/churn_predictions.csv', index=False)