The Broadband Monitoring Programme seeks to provide real-world data sampled from consumer Internet services so that Retail Service Provider (RSP) products can be compared by an independent body (the ACCC) and the results are published publicly. The ACCC has commissioned SamKnows to conduct this data collection using ‘whitebox’ devices that are pre-configured and are simply connected to the end user’s router. As one of the participants of this programme, I quickly noticed the data being collected and reported by my whitebox showed my connection performance was implausibly low. I performed network analysis on the traffic to and from the whitebox on my network using wireshark, then inspected the route to each of the servers the device was communicating with. The results suggest the whitebox is using a method to measure network performance that cannot possibly be accurate or meaningful within the defined context – it’s not measuring the Internet performance between my premises and the RSP, it’s measuring my connection’s usable bandwidth to servers on the other side of the planet. If SamKnows have configured all of their whitebox devices like this, the entire Broadband Monitoring Programme’s data is useless. But even if this is only true for some of the participating devices, it still poisons the data set, invalidating the pool of data, which makes it unusable. I've detailed my methodology, what I discovered from inspecting the device's traffic, and outlined my conclusions on my blog. I've passed this on to my RSP, who are investigating further - I'm not the only one who has noticed these unusual results from the SamKnows whiteboxes. I'd be interested to hear from anyone else who's part of the Monitoring Programme, or anyone else has any insight or feedback to add. I'm not an enterprise or carrier network engineer, I work with business networks and servers, so I'm more than happy to receive feedback In principle I believe such an independent monitoring programme is a great idea, which is why I offered to be part of it - if the whiteboxes can be reconfigured by a C&C server, then this could be fixed relatively easily. But it does need to be fixed, and there needs to be far more transparency and accountability from SamKnows, at the very least to the ACCC and similar bodies in other countries (even if they don't publicly disclose what's going on). If their 'whiteboxes' are just blackboxes where no one outside SamKnows knows what they do or how they're doing it, then their data simply can't be trusted. Which makes them a poor fit as a partner for such an endeavour.