The subsidies for coal and gas generation is very hard to find. A lot of politics involved (they fund many politicians political campaigns) and very difficult to accurately calculate as they range from mining the source fuel all (you can't unlink them, without the mining the coal generation wouldn't occur) the way through to the dumping of waste at the end. There are also many direct and indirect costs which are not born by the generation company during the life of the plant, environmental being just one of them.
They aren't really that hard to quantify, it's not like the government have secret accounts that they use to funnel billions the way of the mining companies, just about all advocates of renewables point to the cost of subsidies to mining as a whole and conflate that with the cost of mining for electricity generation, even then the "subsidies" they mention are generally things like the fuel tax credit which isn't mining specific, it applies to all business equally. The reason that they are "hard to find" is that they don't exist in the way many try to insinuate, what does exist is pretty easy to quantify, just pales into insignificance in comparison to renewables. Whether that cost is justified on environmental grounds etc is a different question.
There are heaps of these graphs around. Working out what data is used is more difficult. But they all point to the same thing... as the cost manufacture, installation and maintenance of renewable stations are dropping, coal is very quickly becoming unattractive financially
Renewables (both with and without storage) vs existing coal, existing coal is cheaper
Renewables vs new coal, renewables are probably cheaper.
Renewables with storage vs new coal, new coal is probably cheaper.
The cost of existing renewables and storage continue to fall so that will likely change as time goes by. So yes building new coal-fired power is becoming less financially viable, keeping existing coal-fired power plants online though is still the cheapest form of non-intermittent supply by a fair bit.
Yes and no. They have a large amount of data now to use for predictions. But future aside, we are pretty much at that crossover point now. Storage is also dropping really fast and the battery tech is also getting much better. Give it 5 or so years and battery storage will be quite affordable even for householders. I'm an electronics engineer (Electrical/Electronics, majoring in Electronics) and in my field I have seen many times really expensive tech plummet in price very quickly once it becomes established.
You only have to look at one of your linked articles to see that they got the predicted increase in electricity demand very wrong, even though it would have been based on a large amount of data. I am not saying it definitely will be wrong, just that the further out they are predicting the greater the uncertainty.
No doubt that it will continue to fall, especially due to government incentives for renewables and regulatory impositions for coal.
I think the underlying question is predominantly a "values" one, Advocates of renewables say we need to do our bit to reduce the environmental impact of climate change or the cost will be borne by future generations, detractors say it's virtue signaling that hurts those who can least afford it and that any changes that we make here in Australia will have negligible difference in the global context.