Rainfall Shortages up to May 2018

Rainfall shortage Manilla May 2018

Rainfall shortages now

On this graph the black line with black squares shows Manilla rainfall shortages at the end of May 2018. Shortages are shown for short terms down to one month, and for long terms up to 360 months (30 years).

Extreme shortages

There were no extreme rainfall shortages at this date.

Severe shortages

There were severe shortages in rainfall totals as follows:
Total for one month (May): 1.2 mm, at the 2nd percentile;
Total for two months (April and May): 19 mm, at the 3rd percentile;
Total for three months (March, April and May): 45 mm, at the 4th percentile.

Serious shortages

Some other rainfall shortages were not severe, but serious:
Total for five months: 136 mm, at the 9th percentile;
Total for twelve months: 408 mm, at the 6th percentile;
Total for sixty months: 2765 mm, at the 8th percentile;
Total for seventy-two months: 3358 mm, at the 6th percentile.

General shortage

The first comment and reply below notes the fact that no rainfall total for any period reaches the 50th percentile. This has not happened for seventy years (1947).

Comparing May 2018 with September 2017

The graph also has a grey line showing similar rainfall shortages at Septemer 2017 (See the earlier post “A drought has begun”.). In the following month, October, there were no rainfall shortages, because the rainfall, 84 mm, was far above average. November, December and February also had rainfalls above average.
Since March 2018, shortages have appeared again. By comparing the black line (May 2018) with the grey line (September 2017), you can see that the rainfall totals are now lower for nearly all periods of time. Only four totals are now higher, including the 4-month total.

What are the classes of rainfall shortage?

We need to compare rainfall shortages. The best way is not by how far below normal the rainfall is, but by how rare it is. That is, not by the percentage of normal rainfall, but by the percentile value. As an example, when the rainfall is at the fifth percentile, that means that only five percent of all such rainfall measurements were lower than that.
Once the percentile values have been worked out for the rainfall record, each new reading can be given its percentile value. Percentile values of low rainfall are classed as extreme, severe, or serious.
For a rainfall shortage to be classed as extreme, its value must be at or below the 1st percentile.
A severe rainfall shortage is one that is below the 5th percentile.
A serious rainfall shortage is one that is below the 10th percentile.
A rainfall shortage that is above the 10th percentile is not counted as serious.

Long-lasting rainfall shortages

Rainfall shortages sometimes last a long time. The same classes of shortage are used for long periods, such as a year, as for short periods, such as a month. They depend on how rare such a shortage is on the average, and they all use the same percentile values to separate extreme, severe, and serious rainfall shortages.

Relations Among Rainfall Moments

Six thumbnail graphs of rainfall moment relationships

Twelve-monthly values of rainfall since 1883 at Manilla NSW yield the four moments of their frequency distributions: mean, variance, skewness, and kurtosis. I plotted the history of each moment (when smoothed) in an earlier post.
Here, I compare the moments in pairs. Connected scatterplots reveal the trajectory of each relationship with time.
Some linear and cyclic trends persist through decades, but none persists through the whole record.
The first image is an index to the suite of six graphs of pair-wise relationships that I present below.

Rainfall Variance vs. Mean

Trajectory of Variance versus Mean

Continue reading

Annual Rainfall Extremes at Manilla NSW: V

V. Extremes marked by high kurtosis

Manilla annual rainfall kurtosis

This graph shows how the extreme values of annual rainfall at Manilla, NSW have varied, becoming rarer or more frequent with passing time.
The graph quantifies the occurrence of extreme values by the kurtosis of 21-year samples centred on successive years.

The main features of the pattern are:
* Two highly leptokurtic peaks, showing times with strong extremes in annual rainfall values. One is very early (1897) and one very late (1998).
* One broad mesokurtic peak, in 1938, showing a time with somewhat weaker extremes.
* Broad platykurtic troughs through the 1910’s, 1920’s, 1950’s, 1960’s and 1970’s, decades in which extremes were rare.
All these features were evident in the cruder attempts to recognise times of more and less occurrence of extremes in Parts I, II, III and IV of this series of posts. This graph is more precise, both in quantity and in timing.

Superseded

The results shown in this post are based on sparse data. They are superseded by results based on much more detailed data in the post “Relations Among Rainfall Moments”.

However, kurtosis (the fourth moment of the distribution) does not distinguish extremes above normal from those below normal. It is known that some early dates at Manilla had extremes that were above normal, and some late dates had extremes that were below normal.

Use of skewness

Extremes above normal are distinguished from those below normal by the third moment of the distribution, that is, the skewness.
Manilla Annual rainfall history: SkewnessThe post “Moments of Manilla’s Yearly Rainfall History” shows graphs of the time sequence of each of the four moments, including the skewness (copied here) and the kurtosis ( the main graph, copied above). The skewness function, like the kurtosis function, relates to the most extreme values of the frequency distribution, but to a lesser extent (by the third power, not the fourth).

I have shown the combined effect of kurtosis and skewness on the occurrence of positive and negative extremes in this data set in the connected scatterplot below.

Manilla rain skew vs.kurt

The early and late times of strong extremes were times of strongly positive and strongly negative skewness respectively. As kurtosis fell rapidly from the initial peak (+0.9) in 1897 to slightly platykurtic (-0.4) in 1902, the skewness also fell rapidly, from +0.7 to +0.3.
Much later, in mirror image, values were almost the same in 1983 as in 1902, then kurtosis rapidly rose while skewness rapidly fell, until kurtosis reached +0.9 and skewness -0.3 by 1998.
Between 1902 and 1983, while kurtosis remained below -0.2, the pattern was complex. In the decades of strong platykurtosis (below -0.9) there were extremes of skewness: +0.7 in 1919 and -0.3 in 1968.
Note that the skewness range was as high in times of low kurtosis as in times of high kurtosis, and the same applies to kurtosis range in relation to skewness. Conversely, when either moment was near its mean, the range of the other was not high.


See also:
“Rainfall kurtosis matches HadCRUT4” and “Rainfall kurtosis vs. HadCRUT4 Scatterplots”.

Annual Rainfall Extremes at Manilla NSW: IV

IV. Some distributions had heavy tails

Graph of history of heavy tails in Manilla annual rainfall

This graph is based on applying a 21-year sampling window to each year in the Manilla rainfall record, then adding smoothing. (See “Note about Sampling” below.)

“Heavy tails”

In the previous postI plotted only the most extreme high and low values of annual rainfall in each sampling window. Now, I choose two rainfall amounts (very high and very low) to define where the “Tails” of the frequency distribution begin. These Tails are the parts that I will call “extreme”. I count the number of values that qualify as extreme by being within the tails.
In this post, I recognise heavy tails, when before I recognised long tails.


Back to the prelude “Manilla’s Yearly Rainfall History”.
Back to Extremes Part I.
Back to Extremes Part II.
Back to Extremes Part III.

Forward to Extremes Part V.


Making the graph

The long-term Normal Distribution

The graph relies on the long-term Normal Distribution curve (“L-T Norm. Dist.” in the legend of the graph). That is, the curve that I fitted earlier to the 134-year record of annual rainfall values at Manilla NSW.
Histogram annual rainfall frequency Manilla NSWThe graph is copied here.

I defined as “Extreme Values” those either below the 5th percentile or above the 95th percentile of the fitted Normal Distribution. That is to say, those that were more than 1.645 times the Standard Deviation (SD = 156 mm) below or above the Mean (M = 652 mm). When expressed in millimetres of annual rainfall, that is less than 395 mm or more than 909 mm.
These ‘Tails’ of the Normal Distribution each totalled 5% of the modeled population, making 10% when added together.

The data

For each year’s 21-year sample, I counted those rainfall values that were lower than 395 mm (for the Low Tail) and those higher than 909 mm (for the High Tail). I added the two to give a count for Both Tails. To get a percentage value, I divided by 21.
I then found the ratio of this value to that of the fitted long-term Normal Distribution by dividing by 5% for each tail, and by 10% for both tails together. Ratios above 1.0 are Heavy Tails, and ratios below 1.0 are Light Tails.
That ratio, when smoothed, is plotted on the main graph at the head of the page.

Results

The resulting pattern of heavier and lighter tails, shown above, is similar to that found by using more and less extreme values, shown in the graph copied here.

Graph of history of extremes of annual rainfallAs before, there were less extremes in the 1900’s, 1910’s, 1920’s and 1930’s.
As before, there were more extremes in the 1940’s and 1950’s.
In the 1890’s, the “Tails” graph did not confirm the more extreme values that had been found earlier.

The 1990’s discrepancy

Extremes had been near normal through the last five decades in the earlier graph. By contrast, the “Tails” graph shows extremes in the most recent decade, the 1990’s, that were just as high as those in the 1950’s. Those two episodes differ, however: in the 1950’s only the high tail was heavy; in the 1990’s, only the low tail was heavy.
(For the 1990’s heavy low tail, see the Note below.)

The inadequacy of the data

Continue reading

Annual Rainfall Extremes at Manilla NSW: III

III. When extreme values were more or less extreme

Graph of history of extremes of annual rainfall

This graph is based on applying a 21-year sampling window to each year in the Manilla rainfall record, then adding smoothing. (See “Note about Sampling” below.)


Back to the prelude “Manilla’s Yearly Rainfall History”.
Back to Extremes Part I.
Back to Extremes Part II.
Forward to Extremes Part IV.

Forward to Extremes Part V.


Making the graph

For each year, I have identified the highest and lowest values of annual rainfall in its 21-year sample. I already know the long-term mean annual rainfall at Manilla: 652 mm. From those values I have plotted the height of the maximum value above the mean (red) and the depth of the lowest value below the mean (green). Both may be called “Extreme Values”.
The difference between the maximum value and the minimum value in each 21-year sample is the Total Range. That also is a measure of Extreme Value, which I graphed in an earlier post.
The Total Range is equal to the sum of the two Extreme Values that are plotted. To make it easy to compare the three measures, I have divided the Total Range by two. I have  plotted that value in blue.

Two discordant results

All three measures agree well except at two dates: 1897 and 1980. On both occasions the Minimum Value (green) was not extreme at all, being only about 200 mm below the long-term mean. The Maximum Value in 1980 was rather extreme (about 330 mm above the mean). The Maximum Value in 1897 was the most extreme value that appears on this graph: 475 mm above the mean!
The pattern of this graph is dominated by this single feature. It is due to just one data item: the annual rainfall reading of 1129 mm in the year 1890, which was the highest ever.

The pattern

For extreme annual rainfalls at Manilla, this graph suggests the following:

They were more extreme than usual at the end of the 19th century and in the 1940’s.
They were less extreme than usual from the 1900’s through to the 1930’s.
They have been no more or less extreme than one should expect through all of the last five decades.

Comment

This graph depends on very simple statistics: the maximum, the minimum and the mean. Such a sparse data set is subject to the effect of chance. Also, although this is not obvious, this graph assumes that other features of the distribution of annual rainfall have not been changing, which is not true.

I have more to say on this topic.


Note about Sampling

I chose a 21-year sampling window to be wide enough to contain enough points for analysis, without losing time-resolution, or losing too many years at each end of the record from 1883 to 2016.
The first mid-year of a sampling window was 1893 and the last, 2006.
To remove jumps in the trace on the graph, I then applied a nine-point Gaussian smoothing function. That further reduced the years that could be plotted to those from 1897 to 2002.

Annual Rainfall Extremes at Manilla NSW: II

II. Platykurtic, Bimodal Annual Rainfall

Histogram annual rainfall frequency Manilla NSW

Manilla’s 134 years of rainfall readings yield the graph above. There are several features to notice.


Back to the prelude “Manilla’s Yearly Rainfall History”.
Back to Extremes Part I.
Forward to Extremes Part III.
Forward to Extremes Part IV.

Forward to Extremes Part V.


A ragged pattern

Despite having as many as 134 annual rainfall values, the graph is still ragged. Some of the 20 mm “bins” near the middle have less than 2% of the observations, while others have over 5%. The pattern has not yet become smooth.

It is not near a normal distribution

Rainfall is thought of as a random process, likely to match a curve of normal distribution. On the first two graphs I have drawn the curve of normal distribution that best fits the data.

Smoothed annual rainfall frequency Manilla NSW

In this second graph, I have smoothed out the ragged shape of the plotted data, using a 9-point Gaussian smoothing. You can see more clearly where the actual curve (black) and the normal curve (magenta) differ. The dotted red line shows the differences directly:

The peak is low;
The shoulders, each side of the “peak”, are high;
Both of the tails are thin.

These three features describe a platykurtic curve: one with low kurtosis. This fact makes the highest and lowest annual rainfalls at Manilla less extreme than would be expected in a normal distribution.

[For an explanation of kurtosis, see the post “Kurtosis, Fat Tails and Extremes”.]

Another departure from normality is that the curve is skewed: the tail on the left is shorter than the one on the right. That is a positive skew, but it is small. (By contrast, most of the rainfall distributions for individual months at Manilla have large positive skew. In them, the peak is well below the mean, and a tail extends to rare high values.)

In summary, four of the leading features of the shape of Manilla’s annual rainfall distribution are:

Mean or average: 652 mm per year.
Standard Deviation (measuring spread or scatter): 156 mm.
Skewness: 0.268 (slightly positive).
Kurtosis: -0.427 (strongly platykurtic).


A platykurtic curve matches the Manilla annual rainfall frequency curve to some extent.

The sum of two Gaussian curves gives a much better match.


Fitting a platykurtic near-normal curve

Much of the poor fit of a normal curve to the data is due to the data having a platykurtic distribution. Being platykurtic produces a reduced peak, high shoulders, and thin tails, as was noted.

Smoothed rainfall frequency and a platykurtic curve

In the third graph, I have drawn (in magenta) a new model distribution that is platykurtic. It is a transform of the normal distribution with a weighted sinusoidal correction. The new curve fits much better up both flanks of the data curve. It cannot be made to fit in the peak area between 500 mm and 820 mm.

Fitting a bimodal model made of two normal curves

The shoulders of the smoothed rainfall distribution curve (black) are not simply high; they are higher than the  zone in the middle where the peak would normally be. There is a major mode (peaking at 5.1%) on the left, a minor mode (3.9%) on the right, and an antimode (3.7%) between them.

Smoothed rainfall frequency and a bimodal curve

Continue reading

Annual Rainfall Extremes at Manilla NSW: I

I. Better graphs of Manilla’s annual rainfall and its scatter

Manilla 21-year rainfall medians

Background

The first two graphs  are new versions of graphs in an earlier post, published also as an article in “The Manilla Express” (28/2/17) and in the “North West Magazine” (20/3/17).

In that article, I said:
“This Manilla rainfall record is one counter-example to the snow-balling catalogue of reported extreme climatic events.”
My claim was not well supported. While the two graphs showed that recent annual rainfalls have been normal, with little scatter, they do not show whether there were any extreme events.

However, Manilla’s annual rainfall record can be analysed to show extreme events. This post considers the Total Range within a 21-year sampling window as a measure of extremes. Using that measure, extremes were at their highest in the 19th century, before anthropogenic global warming began.

A following post discusses kurtosis as another measure, with a different result.


Back to the prelude “Manilla’s Yearly Rainfall History”.
Forward to Extremes Part II.
Forward to Extremes Part III.
Forward to Extremes Part IV.

Forward to Extremes Part V.


The two improved graphs

The re-drawn graphs of historical records in this post use a 21-year sampling window, as before. They now have an improved smoothing procedure: a 9-point Gaussian curve. (The weights are stated below.)

1. Yearly Rainfall Totals

The first graph (above) represents the normal rainfall as it changes. The earlier version showed the arithmetic mean. The new version uses the median value (the middle, or 50th percentile value) instead.
The new version is less “jumpy” due to better smoothing. The median varies much more than the mean does. All the same, most features of the shape are unchanged: very low annual rainfall from 1915 to 1950; very high rainfall from 1955 to 1982; normal rainfall since 1983. There are some shape changes: rainfall before 1900 does not plot so high; from 1911 to 1913 there is a respite from drought; the highest rainfall by far now appears from 1970 to 1980.

As before, one can say:
“Present rainfall will seem low to those who remember the 1970’s, but the 1970’s were wet times and now is normal. Few alive now will remember that Manilla’s rainfall really was much lower in the 1930’s.”

In addition, this new version makes the pattern of growth and sudden collapse obvious. Collapses amounting to 100 mm came within a few years after both 1900 and 1978. Growth in the 58 years from 1920 to 1978 came at the phenomenal and unsustainable rate of 33 mm per decade. By the 1970’s, elderly residents of Manilla would have seen rainfall increase decade by decade throughout their lives.
(I noted this pattern of growth and collapse in an earlier post about Manilla’s summer rainfall.)

Manilla 21-year rainfall Inter-quartile Range

2. Yearly Near-Mean Rainfall Scatters

The plot on this second graph is changed only by better smoothing. However, the titles are changed. I realised that the Inter-quartile Range is not a good general indicator of spread or, in this case, of reliability of rainfall (as I had assumed). Inter-quartile Range measures the scatter of values that are close the middle: just the middle 50%. My new title refers to “near-mean” scatter. Any values that could be called “extreme” fall very far beyond the Inter-quartile Range.

Two more measures of scatter

An alternative measure of scatter in data is the Standard Deviation. In normally distributed data, the Standard Deviation extends 34% each side of the median (and mean). The “Standard Deviation Range” then extends from the 16th percentile to the 84th percentile. It includes a much larger proportion (68%) of a population than the Inter-quartile Range (50%) does. However, it also says nothing about extremes, which will lie far out in the residual 32% “tails” of the data.

The broadest measure of scatter is the Total Range from the lowest to the highest value. This measure does include any extreme values that exist in the data.
In the present case, each calculation uses a sample that includes only 21 points. The lowest data point is close to the 5th percentile and the highest data point is close to the 95th percentile of a similar continuous curve.

All three measures of scatter graphed

Manilla 21-year rainfall Total Range, Standard Deviation Range and Inter-quartile Range

Continue reading