What Python libraries are best for data analysis?

The core stack is pandas (data manipulation), numpy (numerical computing), and matplotlib/seaborn/plotly (visualization). For specific tasks: scikit-learn for machine learning, statsmodels for statistics, scipy for scientific computing. According to the Python Data Science Handbook, pandas is the most widely used library, with over 40 million downloads per month as of 2026.

How do I handle large datasets that don't fit in memory?

Use pandas chunks with chunksize parameter: pd.read_csv(file, chunksize=10000) processes data in batches. For very large datasets (>10GB), consider Dask (parallel pandas), Polars (faster alternative), or Apache Spark. You can also use database queries to filter data before loading, or use parquet format which supports column-wise reading.

What's the best way to clean messy data?

Follow this workflow: (1) Inspect data with df.info() and df.describe(), (2) Handle missing values based on your domain (drop, fill, or interpolate), (3) Fix data types with astype(), (4) Remove duplicates with drop_duplicates(), (5) Handle outliers using IQR or z-scores, (6) Standardize formats (dates, strings, categories). Always document your cleaning decisions and validate results.

Should I use matplotlib, seaborn, or plotly?

Use matplotlib for full control and publication-quality static plots. Use seaborn for statistical visualizations with less code (builds on matplotlib). Use plotly for interactive, web-ready charts with hover tooltips and zoom. For data exploration, seaborn is fastest. For dashboards and presentations, use plotly. For academic papers, use matplotlib.

textcodingintermediate

Python Data Analysis Script

Generate Python scripts for data analysis using pandas, numpy, and visualization libraries

Model: Claude Sonnet 4.6Est. tokens: 1,000

Prompt Generator

Generate Your Prompt

Fill in the fields below to generate an optimized prompt based on best practices

Dataset Description *

What data are you analyzing?

Analysis Goals *

What insights do you want to extract?

Data Format *

What format is your data in?

Key Columns *

List the main columns you'll analyze (comma-separated)

Examples

Sales Analysis

Input Variables

datasetDescription:E-commerce sales data with customer info, products, and transactions

analysisGoals:- Calculate monthly revenue - Find top 10 products - Segment customers by purchase behavior

dataFormat:csv

keyColumns:order_id, customer_id, product_name, price, quantity, order_date

hasNulls:

includeVisualization:

vizLibrary:seaborn

Frequently Asked Questions

Best Practices

Critical Best Practices

Be Specific About Requirements

critical

Include exact specifications: language version, framework, dependencies, coding style, and expected behavior. Vague requirements lead to generic code.

Example:

"Instead of "create a function", say "create a TypeScript function using async/await that validates email format using regex""

Source: custom