Music Charts Archive Scraper

A full-stack React application for scraping and visualizing music chart data from the Music Charts Archive website.

Project Structure

MusicCharts/
├── docs/  
├── frontend/          # React frontend application
│   ├── src/
│   │   ├── components/    # React components
│   │   ├── hooks/         # Custom React hooks
│   │   ├── services/      # API service layer
│   │   ├── App.jsx        # Main app component
│   │   ├── index.js       # Entry point
│   │   └── styles.css     # Global styles
│   ├── public/
│   └── package.json
└── backend/           # Node.js/Express backend API
    ├── src/
    │   ├── routes/        # API routes
    │   ├── controllers/   # Route controllers
    │   ├── models/        # Data models & scraping logic
    │   ├── middleware/    # Express middleware
    │   └── server.js      # Main server file
    └── package.json

Features

• Year Selection: Choose different years to view available chart dates
• Date Selection: Browse and select specific chart dates for detailed song data
• Music Chart Display: View ranked songs with title and artist information
• CSV Download: Export chart data as CSV files
• Responsive Design: Mobile-friendly interface
• Error Handling: Comprehensive error states and loading indicators
• Web Scraping: Real-time data scraping from Music Charts Archive

Frontend Components

• YearSelector: Dropdown for year selection
• DateList: Grid of available chart dates for selected year
• ChartTable: Music chart data table with rankings
• DownloadButton: CSV export functionality
• Layout: Main layout wrapper

Backend API Endpoints

• GET /api/health - Health check endpoint
• GET /api/chart/dates/:year - Get available chart dates for a year
• GET /api/chart/data/:date - Get chart data for a specific date

Setup Instructions

Prerequisites

• Node.js (v14 or higher)
• npm or yarn

Backend Setup

1. Navigate to the backend directory:

   cd backend
   

2. Install dependencies:

   npm install
   

3. Create a .env file in the backend directory:

   PORT=3001
   NODE_ENV=development
   CORS_ORIGIN=http://localhost:3000
   

4. Start the development server:

   npm run dev
   

The backend will be available at http://localhost:3001

Frontend Setup

1. Navigate to the frontend directory:

   cd frontend
   

2. Install dependencies:

   npm install
   

3. Start the development server:

   npm start
   

The frontend will be available at http://localhost:3000

API Documentation

Get Available Chart Dates

GET /api/chart/dates/:year

Parameters:

• year (number): The year to get chart dates for (1970-present)

Response:

[
  {
    "date": "2024-01-06",
    "formattedDate": "Jan 6, 2024"
  },
  {
    "date": "2024-01-13", 
    "formattedDate": "Jan 13, 2024"
  }
]

Get Chart Data

GET /api/chart/data/:date

Parameters:

• date (string): Date in YYYY-MM-DD format

Response:

[
  {
    "order": 1,
    "title": "Lovin On Me",
    "artist": "Jack Harlow"
  },
  {
    "order": 2,
    "title": "Cruel Summer [re-release]",
    "artist": "Taylor Swift"
  }
]

Data Source

This application scrapes data from Music Charts Archive, a comprehensive database of historical music charts. The scraper extracts:

1. Chart Dates: Available chart dates for each year
2. Song Rankings: Top songs with their chart positions, titles, and artists

Development

Adding New Features

1. Frontend: Add new components in frontend/src/components/
2. Backend: Add new routes in backend/src/routes/ and controllers in backend/src/controllers/
3. Scraping: Modify backend/src/models/chartService.js for new data extraction
4. Styling: Modify frontend/src/styles.css for global styles

Testing

• Backend tests: cd backend && npm test
• Frontend tests: cd frontend && npm test

Technologies Used

Frontend

• React 18
• Custom Hooks
• CSS3 with modern features
• Axios for API calls

Backend

• Node.js
• Express.js
• Cheerio for HTML parsing
• Axios for web scraping
• CORS for cross-origin requests
• Helmet for security headers
• Morgan for logging

Legal Notice

This application is for educational and research purposes. Please respect the terms of service of the Music Charts Archive website and implement appropriate rate limiting and caching strategies for production use.

License

MIT License