IMDB Movie List

IMDB is an online database of information related to films, television, etc. But the IMDB Movie List filtering options are far more ancient that it should be. The only two main options are Most Popular Movies and Most Rated Movies.

It inspired me to make my own version of the IMDB with real time filtering options. I scraped details of movies from the website of the top 5000 movies with a minimum rating of 7. I also made sure that the movies had a minimum of 1000 voters. This helped me remove movies that had been overrated from my database.

Technologies Used :

  • Python
  • Angular
  • Firebase Realtime Database

The web application is hosted on Github Pages. You can see the live demo here. In the rest of the blog, I’ll narrate how I made the web application.

Step 1 : Scraping

Before I started making the web application, I had to gather details about the movies I would want to display. So, the first step was scraping and its code can be found in scraper.py

To scrape data from IMDB, I used python’s urllib package. I applied filters in the url to display only movies with a minimum rating of 7.

https://www.imdb.com/search/title/?title_type=feature,tv_movie&user_rating=7.0,10.0&adult=include&sort=num_votes,desc&count=250&start=1&ref_=adv_nxt

The html page I scraped had a maximum of 250 movies sorted by number of voters in descending order. Therefore I had to download a total of 20 html pages ( 250 x 20 = 5000). To prevent downloading of these html files multiple times, I saved the data of each page in a text file. This allowed me to parse them later.

Step 2 : Data Parsing

Now that I had the data of all 5000 movies, I had to extract the information I needed. To serve my purpose, I used Beautiful Soup to parse the data. I iterated over every text files I created in the previous step and pushed details of each movie to a list. The code I used for parsing through all the data can be found in parser.py.

Create a Firebase Realtime Database and download its certificate.json file to authenticate the connection from python. You will also need the firebase-admin library. You can find the documentation here. Finally, I uploaded the parsed IMDB movie list to firebase.

Step 3 : Angular Web Application

I created a basic angular project along with a component ‘movie-detail’ and service ‘firebaseDb’. The movie-detail component contained the html for how each movie will be displayed in a list of movie-details. I connected firebase realtime database with my angular application. You can find the tutorial here.

I added some filtering options in a two-way binded form to filter the movies in realtime. I decided to download all the movies at the start of the application to prevent loading time later. You download the source code to my angular web application here.

If you liked this post, feel free to leave your feedback in the comments section below. I would also love to read you suggestions/ critisism.


Ebenezer

Freelancer Web Developer and blogger. Final Year BCA student at Maharaja Sayajirao University of Baroda

0 Comments

Leave a Reply