100 Toasty Tofu(s) – Another Triple J Hottest 100 Predictor

Leave a reply

Update: Think you can do better than my prediction? Prove it by filling out your prediction here: Triple J Hottest 100 Prediction tracker submission. Also, you can look at the leaderboard of predictions over here.

100 Toasty Tofu(s) is another Triple J Hottest 100 Predictor, made for your entertainment with no guarantees what-so-ever.

Since 2012, various people have been predicting the Hottest 100 using social media scrapes and OCR. This started with The Warmest 100 and was continued by 100 Warm Tunas. I’ve long thought it’s an awesome experiment because the conditions are good for using social media as a predictor. Two factors make this a good experiment – the average person is willing to share their hottest 100 votes and the stakes are so low, unlike political elections, that there aren’t hoards of true believers/trolls/Russian government agents trying to manipulate public sentiment.

I use instagram-scraper to scrape the hashtags (the same as 100 Warm Tunas) and then a python script that uses Tesseract OCR to convert them to text. They are then matched with the Triple J song list (PDF) and saved. I removed any duplicate votes I found, that is people who voted for the same songs in the same order when there are greater than 3 songs in the image (a very unlikely occurrence). I figure these are probably the same person uploading the same image twice.

This is an initial cut, there’s still some extra work to do including:

  • Manually add songs that would be in the hottest 100 to the song list
  • Tune the OCR, including doing some pre-processing to images if needed
  • Tune the matching algorithm – currently using Levenshtein distance
  • Do more analysis on voting combinations (e.g are there factions who vote for particular songs together and what can we learn from this).
  • Make the table pretty like the other ones.
  • Make a form for people to upload their own predictions and show a leaderboard as they come in on the 27th.

The results are quite different to 100 Warm Tunas – I seem to be picking up more votes. I’m not sure if this is due to some sort of filtering I’m not doing or just algorithm differences, but we will see if 100 Warm Tunas still is the internet’s most accurate prediction of Triple J’s Hottest 100 for 2017 on January 27!

This table is updated automatically every few hours.
Total number of images: loading…
Total number of duplicates: loading…
Total number of votes: loading…

# Title Artist Votes % Votes Inc dupes %
Loading… Loading… Loading… Loading… Loading… Loading… Loading…

Routing certain IPs over VPN with DD-WRT without IPTables

4 Replies

I decided I wanted to be able to route certain devices on my network over a VPN connection for reasons that I am sure you can use your imagination (geo-restrictions etc). I didn’t want everything to go through the VPN because that would slow down my connection for things I didn’t need it for.

It’s worth noting that before this year you could just use some fancy DNS tricks to route only traffics from a certain domain over your VPN, but I found this failed on devices with hard-coded DNS (like the chromecast or the Android Netflix app).

Media devices like Smart TVs and Chromecasts can’t run OpenVPN so it has to be done on the router. If you want to do this, make sure your router is up to scratch. Encryption uses processing power which most routers lack. You want to be getting minimum 5Mbps with a recommended 10 for this to be usable. I forked out for an R7000 which is probably overkill. Another option is choosing a VPN provider (or setting up your own) that enables you to use weaker encryption – the idea being that it doesn’t really matter that the NSA can snoop on your netflix, it’s up to you.

My first idea was to have a separate WLAN (Wireless LAN) with it’s own subnet and DHCP and route all connections through the VPN. That way you could choose to go over the VPN just by switching networks. I’m sure there is a way to do this, but I couldn’t get it working with my limited knowledge of dd-wrt and iptables and the like. Issues I ran into went from not being able to access the other local subnets (which I wanted to for things like Plex) and just generally getting it to play nice.

So I scrapped that idea and moved onto the next. Give every device you want to route over the VPN a static DHCP lease (i.e their IP doesn’t change) and then use the Policy Based Routing field to tell the router to route internet traffic over OpenVPN. This worked perfectly. The only catch is with Chromecast your mobile device also has to be over the VPN or you won’t be able to see the geo-restricted content. If you don’t always want your phone to go over the VPN for wifi then you could use a cheap tablet as a Chromecast remote OR install OpenVPN on your phone and only connect when you want to access geo-restricted content.

OK so here is how you do it. These instructions assume that you have set up your router to the point of having an internet connection and a single subnet with DHCP turned on.

  1. Put your devices on a Static Lease
    Go to Services > DHCP Server > Static Leases
    Add each device one at a time, pressing save and apply after each time. Note that the hostname doesn’t really matter here, MAC Addresses do and I found some of the hostnames made nothing resolve so if there are any special characters in your hostname just name it something else.
  2. Set up OpenVPN
    Instructions will be different for each provider. OpenVPN is under Services > VPN > OpenVPN Client.
    The only deviation will be that you don’t want to redirect your gateway so remove redirect-gateway from the additional commands
  3. Add you IPs to OpenVPN Client config
    Under Services > VPN > OpenVPN Client > Policy based Routing add each IP in the form of X.X.X.X/32 with one per line. I put both my Chromecasts and my TV on it as well as my cheapo tablet that I use solely for Plex/Netflix.
  4. Bingo. You’re done. No telneting, no iptables no messing around.

I wish I had found this earlier and maybe I would have saved myself some messing around.

University Portfolio

Leave a reply

I think one of the most important parts about studying computer science or software engineering at university is that it gives you the ability to slowly build a portfolio of small pieces of code which can demonstrate what you are capable of. I have embarked on a project over the last month to collate all of my significant university programming assignments. This is a general snapshot of what you learn in a CS degree these. If you are a student – don’t plagiarise my code for your assignments, you’ll get caught and lose your marks. It also violates the license on the code (GPL v2.0) where you must reference the author. If you’re a lecturer – I hope this doesn’t bother you, you should really be changing the assignments every semester anyway to allow for things like this :).

The projects are listed below. I haven’t included all projects as many were fairly trivial and not all computing units assess with programming assignments (much to my annoyance). The code has not been updated since it was first written, so please take note that generally my style of coding has evolved since first learning 4 years ago:

First Year

Second Year

Third Year

 

 

Youtube Song Downloader

Leave a reply

A little project I worked on for the last day was getting a program to make downloading a list of songs off youtube easier. Initially it was just going to be command line, import from a CSV file. But this only works when you know the first hit will be the correct song. So I decided to flesh it out into a GUI.

This was inspired by looking at the sexy lists on the Triple J Hottest 100 Wikipedia page and deciding there should be a way to grab all those songs easily.

Unfortunately, the Google API restricts this kind of thing. But I’m sick of this project now, so here it is. You’re limited to downloading about ten songs at a time, otherwise you get service abuse messages. Hopefully in the future I can find a better method of searching for songs.

Youtube Downloader

Blackboard Scraper now has stand-alone download option.

Leave a reply

I have finally gotten around to making the Blackboard Scraper stand alone so you no longer need to install lots of different things to get it working.

Hopefully this makes it easier for non-computing students to access and use. It still only works on Curtin’s blackboard system, however a UWA one is in the works.

Head to the Blackboard Scraper page to give it a whirl.

The Mystery Box Solver

Leave a reply

Uther Party is a custom mini-game map for the game Warcraft 3. Our local university LAN club has a weird obsession with it. Basically there is a box with a number on it, and the game is to not be holding the box when it lands on 0. You pass the box to the person next to you by pressing either Q, which decrements it by 1 or W which decrements it by 2. If the box is on 1 and you decrement it by 2, it explodes before you pass it.

Uther Party 1

Uther Party

The game is pretty frustrating, so I decided to make a script to tell me the optimal choice to make. The script works by generating a minmax tree of all possible choices. Each leaf has an array the size of the amount of players, which has a 1 in the field if the player doesn’t die in that playthrough or a 0 if they do. It then goes up the tree with each level of the tree corresponding to a different players turn which attempts to maximise their index in the array. If the two choices have the same chance of death, it takes the average of both children’s chances.

The solver works pretty well, however it assumes that the players aren’t colluding and are only looking to maximise their own score which is a fair assumption, but may not be the case 100% of the time.

Mystery Box Solver

You can download the solver here (it has prompt, command and GUI modes). It only requires python, but optionally can use ete2 to print out a graph of the tree if you are really sadistic.

If you want to try it out, I’ve also made a module for the ComSSA IRC bot to facilitate a text based game of mystery box.

Scanning barcodes and QR Codes on Android with a broken back camera

3 Replies

So for reasons I won’t get into, I have a faulty main camera on my Samsung Galaxy S III.

It turns out Snapchat still allows me to take pictures and save them without crashing, due to it having front camera on by default. But I found myself needing a barcode scanner or QR Code scanner for 2-Factor authentication.

Screenshot_2014-07-07-18-35-18

Most barcode scanner apps, which automatically boot into scanning mode will just crash straight away. I spent quite a while trying poorly written bar code scanners (even the one Google Authenticator suggests) which just crash if the back camera is broken.

Eventually I found a solution. QuickMark Barcode Scanner doesn’t crash right away and allows you to switch to your front camera. It has a pretty decent UI, and they also have a pretty good Windows app for scanning barcodes on your screen. Good job on the developers for handling erroneous situations gracefully.