Cycling into the wind

So now I've merged the wind data that I was using with the thorn plots and the cycling data on the Tableau public dashboard. It uses an average wind speed and direction based on the two metar observations nearest to the mid-time of the ride. 

I wondered how to show wind direction, but in the end a simple grouping by quadrant was easy to do and good enough to separate the data into meaningful groups.

The effect of windspeed on the way home, when the wind is in the SW is pretty clear, I think.

I just wish I could get a sensible data between start&end calculated field in the title. Could not get this to work in Tableau.

Routes of A350 and B787 - updated visualisation

The latest version of the routes map has been updated to use great circle links between airports. There seem to be 2 options for this - getting Tableau or R to do the work. Getting R to do the work was easier for me (just an application of gcIntermediate()) than getting to grips with interpolation functions in Tableau. The cost is that you pass a lot of extra data. I suppose I could have used 2 files and done a join in Tableau instead, but the volumes are not huge. 

There are tableau workbooks around to do the work, but from tableau public it doesn't seem to be possible to open and copy the calculations as suggested.

Tableau visualisation of routes by new aircraft types: B787 and A350

The previous post gave a glimpse of the technical, R, side of processing some data compiled by spotters of where the new aircraft, B787 and A350, are flying. The R took a while to get right, but once the data were compiled, getting them into Tableau was a breeze.

There's a beta version here. Beta, mostly in the sense that there are still 1 or 2 glitches in the geo-coding of the data (so in R), but I thought it might be easier to see them given a Tableau map. I like the way the graph on the left acts as a filter for the map - for example comparing the routes of ANA and Japan Airlines.

And beta too, probably because the viz could also be improved. Comments welcome.

 

Interpreting loosely formatted aviation text

This is a short, technical account of using the NLP package of R to interpret some 'loosely-formatted' text about services offered by airlines. In the text, airports are sometimes mentioned by IATA 3-letter code, sometimes by a text name, and not always the same name.

It took a while to get my head around how NLP (and OpenNLP) work. The manual entries are accurate, but that's with hindsight. So I hope the following helps.

I built a natural entity labeller, but first had to work out how the entity annotator syntax works. This is an example of perhaps the simplest possible one, that labels everything as the entity 'all'.

identity_tokenizer <- function(s) {
  #log all words together as one single entity, type 'all'
  #the extent is measured here in words, not characters
  Annotation(1, "all", 1L, length(s))
}

identity_entity_annotator <- Simple_Entity_Annotator(identity_tokenizer)

For each line of text, eg "Delhi DEL – HKG Hong Kong – KIX Osaka Kansai (Commenced)" I used the sledgehammer approach of OpenNLP

AnnSentWord <- list(Maxent_Sent_Token_Annotator(),

                           Maxent_Word_Token_Annotator())
el <- NLP::annotate(as.String(r), AnnSentWord)

        
to get r annotated as sentence and words. Then I invoked my new annotator, which looks for airports (3-letter IATA codes, or 4-letter ICAO codes, or 1-n word names) and labels these as 'airport', with the ICAO code as a 'feature'.

el2 <- NLP::annotate(as.String(r), AP_entity_annotator, el)

        
For some reason, adding this to 'pipe' of sentence and word annotation always failed. The annotator itself is fairly mechanical, using lookup environments (see list2env()) since the guidance on-line is that these are relatively fast - and in my own tests they were about 30 times faster than simple subsetting of dataframes.

The answers are deterministic, not probabilistic, but are based on a likelihood order: 4-letter and 3-letter upper case matches are unlikely by chance; longer name matches 'Santiago de Chile' take precedence over shorter ones 'Santiago'; airports with more traffic are more likely than quieter ones. On average, I had about 4 names per airport.

It's not the most sophisticated application of NLP, but you have to start somewhere!

Thorn Plot, for visualising head winds, final version

Continuing the short series on plotting headwinds, perhaps you prefer the 'thorn plot' version, in which the relative frequency of wind from a particular direction is represented both by transparency of colour and also by the width of the triangle.

For this version, I finally remembered to normalise the number of observations between locations. There are more records for some airports than others, but this is irrelevant for the graph, so I rescale.

Final daisy plot for visualising head wind when cycling.

Got there in the end. I know he guidance is that colour is not the best way to show variation, but this 'daisy plot' works for me, and is less untidy than the 'thorn' variation.

The curved line comes from those explicitly drawn by ggplot in the geom_poly statement, so repeating the start point of the poly gave me a balanced curve on both sides (which gives a clue as to how to get straight lines, but I'm sticking with this version).

Quite a lot of ggplot in the end:

ggplot(y8p , aes(x, y, group = windHead, alpha = numObs)) + 

  scale_alpha(range = c(0.3, 1), breaks = seq(0, 24000, by = 4000)) + 

  geom_polygon(fill = "blue") +

  xlab("Angle = Wind heading (deg) ") + 

  ylab("Mean wind speed (km/h)") +

  labs(alpha = "Observations") +

  coord_polar(start = -pi/8) + 

  scale_y_reverse(limits = c(maxKMH, 0)) + 

  scale_x_continuous(limits = c(-22.5,337.5), breaks = NULL) +

  geom_label(aes(x=0, y = maxKMH, label = station), colour = "black",

             show.legend = FALSE) +

  #theme(legend.position = c(1,0), legend.justification = c(1,0)) + 

  theme(strip.background = element_blank(),

        strip.text.x = element_blank()) +

  facet_wrap( ~ station)

For which, the y8p data looks like this. Note that I've created 4 lines for each real line of data - and x and y give the 4 points used in the polygon for each real point of data.

  station windHead   windKPH numObs     x         y

    <chr>   <fctr>     <dbl>  <int> <dbl>     <dbl>

1    EBBR        0  9.167982   7190   0.0  9.167982

2    EBBR        0  9.167982   7190  22.5  0.000000

3    EBBR        0  9.167982   7190 -22.5  0.000000

4    EBBR        0  9.167982   7190   0.0  9.167982

5    EBBR       45 10.177946   7358  45.0 10.177946

6    EBBR       45 10.177946   7358  67.5  0.000000

More Pies, Roses and Thorns

Continuing the hunt for ways to show headwind, rather prettier are these version, using an explicit geom_polygon to construct triangles.

In the first version, I'm using both transparency and thickness to code the number of observations (frequency that the wind was in this direction). I think this looks like thorns. Haven't worked out why the axis transform should put the twist on the triangles. Suggestions welcome!

In the second version, I've stuck to fixed width. This is more like the diaphragm of a camera, or a daisy.