A lot of Americans have trouble with Australian vs. British, but I've learned the difference after being browbeaten by my BFF (who has an ear for such things and spent 6 years living in Europe). I can't tell Aussie from New Zealand. Irish and Scottish most Americans can tell from the other dialects.
Afrikaans sounded a lot like Australian to me, but the last time I heard it spoken was a good ten years ago and my ear was far less trained at that time.
It does annoy me sometimes when people use the term 'British accent'. You've got this this (Jay-ell you'll like that one) this this this this and it literally differs every 10 miles. People can tell which village I'm from in the local area
It's fine saying 'he's got a British accent' or whatever but when people are like 'I'm imitating a British accent' and then just speak like the Queen it fucks me off no end
It's the same in the US. Northwestern Ohio, where I grew up, has a type of accent that is distinct from the Michigan accent, which is different from that heard in Columbus OH, which is very different (to my ear) from the Cincinnati accent, which is similar to but still distinct from how they speak across the river in Kentucky. In Michigan, you've got at least 3 accents: rural south, Detroit, and Upper Peninsula. And even within a city -- Detroit, for instance -- the white people who live there speak significantly differently from their black and Hispanic neighbors; the accent is based as much on culture and social class as anything else. So yeah, "American accent" is remarkably reductive, but it serves its purpose for discussions like these.
Unfortunately, 97% of the people in the US who claim to be able to imitate a "British" accent are between the ages of 14 and 20, and their main exposure to European culture has come in the form of repeated viewings of "Monty Python and the Holy Grail." It's tragic, really.