Exploring the 117th United States Congress: Insights and Analysis

Introduction

I’ve long been curious about legislative data—how bills become law and what drives the process from introduction to passage. More importantly, I wanted to explore ways to make this data accessible and transparent, helping people better understand how Congress works.

This project analyzes the 117th Congress using data scraped from Congress.gov. While my ultimate goals include geographic analysis and predictive modeling of voting patterns, I’m starting with fundamental exploratory analysis.

Today, I’ll walk through the data collection process and share key insights about legislative patterns, party dynamics, and policy areas.

Data Collection

My primary source is Congress.gov, maintained by the Library of Congress. I focused on the 117th Congress (2021-2023), collecting data on resolutions and joint resolutions while omitting amendments and concurrent resolutions.

Data collected:

Technical Implementation

Building the web crawler was straightforward thanks to Congress.gov’s well-organized structure. I used Python with BeautifulSoup and Selenium to handle the site’s dynamic content loading.

Key technologies:

• Python for core logic
• Selenium for dynamic page loading
• BeautifulSoup for HTML parsing
• Regex for text post-processing

To respect the site’s bandwidth, I added 5-second delays between requests, resulting in a 3-day crawl time. The crawler and processed data will be available on GitHub for public use.

For each bill, I queried two pages:

• All info page: https://www.congress.gov/bill/117th-congress/{bill_type}/{bill_id}/all-info
• Text page: https://www.congress.gov/bill/117th-congress/{bill_type}/{bill_id}/text?format=txt

The parsing process involved targeting specific HTML elements and implementing basic caching to avoid redundant requests.

Key Findings

This analysis focuses on high-level patterns rather than cross-feature relationships or bill text analysis. The insights are organized by the main variables tracked.

Legislative Outcomes

What matters most: which bills get introduced and which become law?

Each bill has a tracker status indicating its position in the legislative process. The eight possible statuses can be grouped into three meaningful categories:

• Introduced: Bills introduced but never voted on
• Stalled: Bills that saw votes but didn’t become law (since the 117th Congress ended, these effectively died)
• Law: Bills signed by the President

Key insights:

• Only 7% of introduced bills ever receive a vote
• Of bills that receive votes, 36% become law
• Overall, just 2% of introduced bills become law

Sponsor Analysis

The bill sponsor—the primary member who introduces legislation—provides insights into party and geographic patterns.

Party Breakdown

Party comparison:

• Democrats: 7.5% of bills moved beyond introduction; 2.6% became law
• Republicans: 5.5% of bills moved beyond introduction; 2.1% became law
• When bills do advance, Republicans have a slightly higher success rate (38% vs 35%)

Geographic Distribution

Top 10 states by bills introduced:

Per-representative normalization reveals different patterns:

Top Individual Sponsors

Most prolific legislators by bills introduced:

Effectiveness score (laws enacted / total bills):

$$ \text{effectiveness} = \frac{\text{bills that became law}}{\text{total bills introduced}} $$

Policy Focus Areas

Each bill is assigned a primary policy area. Here are the most active areas by legislative outcome:

Notable patterns: Health dominates introductions but has lower success rates, while government operations and armed forces bills are more likely to become law.

Next Steps

This analysis reveals clear patterns in congressional activity: most bills die in committee, party affiliation influences success rates, and certain policy areas have higher passage rates than others.

Future work will explore:

• Committee dynamics and voting patterns
• Geographic analysis of state-level interests
• Bill text analysis using NLP techniques
• Predictive modeling for bill outcomes

Update: I’ve now written about Congressional Bill Policy Area Classification, which uses machine learning to automatically categorize bills by policy area using 48K+ bills from three Congresses. See the complete Congressional Data Analysis series for the full learning path.

The complete dataset and analysis code will be made publicly available to encourage further research into legislative transparency.

Have thoughts or questions about this analysis? I’d love to hear from you!