Data Sources
Complete list of news sources, government databases, and social media platforms targeted for scraping.
News Publications (Tamil & English)
Section titled “News Publications (Tamil & English)”| Source | Type | Scrape Method | Language |
|---|---|---|---|
| The Hindu | Daily newspaper | RSS | English |
| Times of India | Daily | RSS | English |
| The New Indian Express | Daily | RSS | English |
| Dinamani | Tamil daily | Playwright | Tamil |
| Dinamalar | Tamil daily | RSS + Playwright | Tamil |
| Dina Thanthi | Tamil daily | Playwright | Tamil |
| The Hindu Tamil | Tamil daily | RSS | Tamil |
| BBC Tamil | Tamil news | RSS | Tamil |
| OneIndia Tamil | Digital news | Playwright | Tamil |
| ABP Nadu | Digital news | Playwright | Tamil |
| Zee News Tamil | TV+Digital | Playwright | Tamil |
| News18 Tamilnadu | Digital | Playwright | Tamil |
| Times Now Tamil | Digital | Playwright | Tamil |
| NewsTamil 24x7 | Digital | Playwright | Tamil |
| Polimer News | TV+Digital | Playwright | Tamil |
| Puthiya Thalaimurai | TV+Digital | Playwright | Tamil |
| Thanthi TV | TV+Digital | Playwright | Tamil |
| Vikatan | Weekly magazine | Playwright | Tamil |
Government & Official Sources
Section titled “Government & Official Sources”| Source | Data Type | Format |
|---|---|---|
| Tamil Nadu Police | FIRs, press releases | Web |
| TN Government (tn.gov.in) | Press releases | Web |
| NCRB (ncrb.gov.in) | Crime statistics | CSV/PDF |
| National Commission for Women | Case reports | Web |
| Madras High Court | Judgments | Web |
| TN Police Twitter/X | Real-time alerts | API |
| District Administration | Local news | Web |
Social Media & Video
Section titled “Social Media & Video”| Platform | Data Type | Method |
|---|---|---|
| YouTube | Tamil news channels | Data API + transcript |
| Twitter/X | Trends, breaking news | API |
| Community pages | Public scraping | |
| Telegram | TN news channels | Public channel scraping |
YouTube Channels for Transcription
Section titled “YouTube Channels for Transcription”| Channel | Focus | Language |
|---|---|---|
| NewsTamil 24x7 | Breaking TN news | Tamil |
| BBC Tamil | In-depth analysis | Tamil |
| Vikatan TV | Investigations | Tamil |
| Polimer News | Rural TN | Tamil |
| Thanthi TV | Crime, politics | Tamil |
| Puthiya Thalaimurai | TN politics | Tamil |
| ABP Nadu | Digital-first | Tamil |
Priority for Phased Implementation
Section titled “Priority for Phased Implementation”Phase 1 (MVP)
Section titled “Phase 1 (MVP)”RSS feeds from 5 major sources + YouTube transcripts
Phase 2
Section titled “Phase 2”All news publications + government sources
Phase 3
Section titled “Phase 3”Social media + full YouTube channel coverage