New MH370 breakthrough tracking technology passes another validation test.
I have used an MH370 search aircraft from the Royal New Zealand Air Force on March 28th, 2014 to help validate this new technology called GDTAAA (Global Detection and Tracking of Aircraft Anywhere Anytime).
This new system is based on the Weak Signal Propagation Reports (WSPR) pronounced ‘whisper’ and promises to give a new search for MH370 a more precise location of the Boeing 777. This new analysis aligns with the satellite and drift modelling that points to an end point of MH370 at around 34.5°S near the 7th Arc.
This validation report is one of many planned to build a compelling body of work before the final analysis of the MH370 impact zone.
The report can be downloaded here
@Richard,
On 25 May 2021 at 12:47, You wrote “GDTAAA has no aircraft identification capability”.
On 24 May 2021 at 15:52, You wrote “Here is a test of the GDTAAA output using the flight path of an AMSA Challenger jet sent to help rescue the Indonesian fishermen after their boat started to sink in heavy weather. The aircraft departed from Perth Airport at around 03:53 Perth Time (PT).”
Methinks you were mistaken about the aircraft type, but the your stated timing “at around 03:53 Perth Time (PT)” is fairly specific. May I ask how did you obtain this take-off time estimate ? Have you evidence that the aircraft left Perth Airport, and not perhaps RAAF Pearce ? Knowing the airport for certain may perhaps aid in identification of the aircraft type.
Whether or not it identification of the aircraft type is of relevance maybe a moot point. The only advantage I can see is if the flight you are currently tracking undergoes some behaviour as witnessed by GDTAAA which would be unlikely for the actual type. (Such as unlikely endurance, for example.)
@George G,
I do not believe I was mistaken about the aircraft type, it is possible that it was RSCU550 on an earlier sortie. It may also have been RSCU660 which is the same aircraft type.
I decided to start looking at 04:00 UTC and discovered an aircraft detection just after take off about 7 minutes out from Perth Airport. I agree I should go back and look each 2 minutes prior to 04:00 UTC to see exactly which flight path it took and which airport it came from Pearce AFB or Perth Domestic Airport.
I am not saying that the video and photographs at night were taken on the same sortie as the video and photographs of D2S2 at 14MAY2021 05:43:48 UTC and 14MAY2021 07:45:57 UTC. To the contrary the IR video and photographs at 13:47:25 UTC around 8 hours later may have been the same aircraft but definitely a separate and subsequent sortie.
@David, @George G,
A guy in Sydney going by the name of “scan Sydney” picked up a RAAF Hercules C-130 A97-442 call sign RSCU211 leaving Richmond AFB towards Western Australia. More interestingly he picked up a satellite ADS-B signal from a Cobham SAR Services AMSA Challenger jet VH-XNF and call sign RSCU550 over the southern Indian Ocean.
The position given was 31° 40′ 29.2″S (-31.674778) 108° 47′ 48.2″ E (108.796722) on 14th May 2021 at 14:19:30 Sydney local time (04:19:30 UTC). The aircraft was outbound to the search area on a track of 273.0 °T at 31,211 feet and with a ground speed of 412 knots. You can see the date time stamp of the Tweet is 14th May 2021 06:23 AM Frankfurt time (UTC+2) where I am based and viewed the Tweet, which is 14:23 PM Sydney time were it was posted just 3.5 minutes after downloading the satellite ADS-B data at 14:19:30 local time.
https://www.dropbox.com/s/u7hsp6cfmhguvm2/Twitter%20ScanSydney%2014MAY2021.png?dl=0
Here is a 10 minute snapshot of the WSPR data from just before to just after 04:19:30 UTC showing the aircraft position according to the satellite ADS-B data.
https://www.dropbox.com/s/9vxe7ql51rgjso3/GDTAAA%20AMSA%20IN%2014MAY2021%201212%20PT%20A.png?dl=0
https://www.dropbox.com/s/w20clc0p5evqhgr/GDTAAA%20AMSA%20IN%2014MAY2021%201222%20PT%20A.png?dl=0
The timing of the flight of the AMSA Challenger VH-XNF call sign RSCU550 fits with taking the video and photographs of D2S2 at 14MAY2021 05:43:48 UTC and 14MAY2021 07:45:57 UTC before returning to Perth Domestic Airport after a 6 hour sortie.
The flight I have been tracking up until now was 7.5 hours earlier and it is possible it was the same aircraft on an earlier sortie. The sorties are usually 6 hours long with a 1.5 hour gap back in Perth for crew change and refuelling (3 sorties in 24 hours).
In the same Tweet “scan Sydney” mentions a 2nd AMSA Challenger jet VX-XND call sign RSCU 660 and a RAAF Poseidon P-8A A47-009 call sign RSCU251.
If you are in agreement, I will switch to tracking the AMSA Challenger on the 14th May 2021 04:10 UTC sortie to the search area and see if it aligns with the video and photographic evidence we have from @David’s AMSA link.
@Richard,
I’d prefer you don’t change horses Mid-Ocean.
@Richard. Switch to 04:10 sortie: agree for my part, good idea .
@David, @Richard,
VH-XNF operating as RCSU550 left Perth at 11:23 AWST (03:23 UTC) and returned 17:47 AWST. The 29 second video “AMSA Challenger Vision – Vessel in distress” starting at 05:43:50 UTC very likely was taken not long after arrival at and descent to the site of the vessel in distress.
VH-XNF then again left Perth at 19:23 AWST for the evening sortie, returning back to Perth at 01:34 Saturday 15th.
Source: Flight Aware, whose data seems a little sparse in the critical region of the west of Perth.
VH-XNF flew in from Cairns, for the above tasks.
VH-XNC operating as RCSU440 and presumably based in Perth had made an earlier flight to the region on Thursday evening returning to Perth 02:54 AWST Friday morning.
VH-XND operating as RCSU660 conducted one sortie, from 23:13AWST on Friday 14th returning 05:30 on Saturday morning, essentially taking over from VH-XNF.
VH-XND had flown in from Essendon (in Melbourne Vic.).
VH-XNE appears not to have been involved, remaining in Cairns.
Someone with a paid subscription to FlightAware or flightradar24 or other alternative source may be better able to find more details.
@Richard,
I would prefer that you don’t change horses midstream. I see advantages in continuing tracking a whole flight, including psychological. By the time you have completed all aspects of one whole flight you may have found need for and made some (more) refinements as you go. For this flight, for all these AMSA and associated Search and Rescue flights, you are using http://www.wsprnet.org data (to be called secondary data) similar to the “Historical” data available from March 2014.
(That is my assumption.)
Advantageously the for the “current” SAR flights additional more detailed data should be available. Analysis of this, subsequent to derivation of the GDTAAA tracks, may assist in identifying pitfalls in use of only the secondary data for any tracking of the March 2014 flight.
@George G. Your 26th 01:14. Great work. That does clear things up.
AMSA’s advice re use of ‘air trackers’ has proved sound if indirect, implicitly updating without admitting.
@George G, @David,
Many thanks for your research and analysis regarding the various SAR flights. @Rob has also provided me with his recent WSPR detections in the SIO and independently identified several of these SAR flights.
Here is a table of the 9 sorties that we all working together have identified so far:
https://www.dropbox.com/s/cs36jai4s8y7sq5/AMSA%20SAR%20D2S2%20Rescue%20AIrcraft%20and%20Sorties.png?dl=0
Here is the same data in Excel:
https://www.dropbox.com/s/a6xqha0n1u9wclw/AMSA%20SAR%20D2S2%20Rescue%20AIrcraft%20and%20Sorties.xlsx?dl=0
Here is all the FlightAware flight tracking data for each of these sorties in a folder:
https://www.dropbox.com/sh/rmxrtzdz5wy9ms1/AACB2FZt4WWmh2C0IqciYu48a?dl=0
From the table and FlightAware data you can see there were three AMSA Challenger jets involved as @George G has confirmed, one RAAF Poseidon P-8A and one RAAF Hercules C-130J as “scan Sydney” has additionally confirmed. The AMSA media release however mentions a second RAAF Poseidon P-8A aircraft.
The first aircraft I was tracking appears to be the AMSA Challenger VH-XNC RSCU 440 or alternative aircraft (sortie 2) although it is not in FlightAware. There is a gap in the activity log for this aircraft in FlightAware between the D2S2 SAR flight on Thursday 13th May 2021 18:54 UTC (sortie 1) and Wednesday 19th May 2021 00:46 UTC. The timing of sortie 2 would fit the arrival of the AMSA Challenger VH-XNC RSCU 440 aircraft back from sortie 1 on 13th May 2021 at 18:54 UTC in time for a departure one hour later after refuelling and crew change at 19:53 UTC.
The second aircraft I was tracking is confirmed to be AMSA Challenger VH-XNF RSCU 550 (sortie 4) and is in FlightAware with a gap in the data when out of reach of ADS-B receivers in the middle of the southern Indian Ocean (SIO). There is no gap in the activity log for this aircraft. On this sortie the video and photographs at 14MAY2021 05:43:48 UTC and 14MAY2021 07:45:57 UTC were taken.
AMSA Challenger VH-XNF RSCU 550 (sortie 6) is in FlightAware with a gap in the data in the SIO. On this sortie the video and photographs were taken with Infra Red (IR) at night time on 14MAY 2021 13:47:25 UTC.
The third aircraft I might track is the AMSA Challenger VH-XND RSCU 660 (sortie 7) and is in FlightAware with a gap in the data when out of reach of ADS-B receivers. There is no gap in the activity log for this aircraft.
I am disappointed that there is a gap of 6 days in the activity log for the AMSA Challenger VH-XNC RSCU 440 in FlightAware. The aircraft might have been unserviceable and grounded if that timeframe.
I am interested to follow both the AMSA Challenger VH-XNC RSCU 440 or alternative aircraft (sortie 2) through to the search area as well as the AMSA Challenger VH-XNF RSCU 550 (sortie 4) through to taking the videos and photographs and its departure for Perth.
All the WSPR data I am using in 2014 or 2021 comes from https://wsprnet.org/ so this is a good validation test for the SIO around the search area for MH370. MH371 will be a good validation test in a completely different area. Between these various tests I am aiming to cover different locations and different time frames.
I am still hopeful that the flight data from the earlier AMSA MH370 SAR sorties in 2014 will be made available by the ATSB to further validate my earlier GDTAAA output on 28th March 2014.
@All
In researching the aircraft involved in the D2S2 SAR I found this statement from the Australian Airforce dated 2017: “RAAF P-8A Poseidon assists in its first civilian search and rescue response”.
“The Royal Australian Air Force’s (RAAF) latest maritime surveillance and response capability – the P-8A Poseidon, has assisted with the co-ordination of a search and rescue response, approximately 50 kilometres south of Mount Gambier, Victoria. While on a training mission from its home at RAAF Base Edinburgh in South Australia, the P-8A Poseidon, call sign Blackcat 20, intercepted a mayday call from a distressed vessel, the MV Port Princes, around 1 pm on Tuesday, 16 May 2017.”
A RAAF Poseidon P-8A also assisted with the search for D2S2 on 13th May 2021 departing from Edinburgh AFB (YPED) at 23:33:25 UTC (14th May 2021 09:33:25 Local Time). Now 4 years later, the RAAF are certainly keeping busy with SAR responses.
https://www.airforce.gov.au/news-and-events/news/raaf-p-8a-poseidon-assists-its-first-civilian-search-and-rescue-response
@All
Live data on FlightAware is subject to a standard data processing delay of up to two minutes. This is not applicable to ADS-B data received directly from a satellite or raw ADS-B data received directly from ATC.
The AMSA Challenger VH-XNF call sign RSCU550 departed Perth International Airport on 14th May 2021 at 03:23:59 UTC. The last ATC ADS-B from FlightAware on the outbound flight to the SIO was at 04:02:47 UTC around 248 nm due West of Perth Airport at 31.7755°S 111.1203°E.
“scan Sydney” picked up the same flight at 04:19:37 UTC at 31.674778°S 108.796722°E, but this time the ADS-B data came via satellite.
I tracked the aircraft for 24 minutes from 04:00 UTC to 04:24 UTC and compared the GDTAAA position to the ADS-B ATC data and the ADS-B SAT data.
In addition to the 18 nm accuracy of WSPR signals due to the 6 character Maidenhead Grid code, there is up to a 2 minute system latency in FlightAware data. In this test I measured a latency of around 1.57 minutes.
However as the test was running I noted the aircraft was in the midst of a step climb. At 04:00:00 UTC the aircraft was at 30,000 feet with a ground speed of 434 knots. At 04:19:30 UTC the aircraft was at 31,211 feet and 412 knots. I assume the aircraft eventually levelled off at 32,000 feet and the ground speed increased again to around 434 knots. I will find out with further tests.
There is another limitation to the GDTAAA calculations of ground speed and track. Previously I have stated that the ground speed and track require position indicators over a period of time. I presented earlier an hour of tracking data for another AMSA SAR flight and the accuracy of the ground speed and track calculations was demonstrated in level flight at cruise speed. However, when an aircraft is climbing or turning then infrequent position indicators will lead to incorrect results. In this test there were 6 progress indicators in 24 minutes, of which 2 are possibly position indicators. The GDTAAA ground speed and track calculations will improve with the GDTAAA output for the subsequent part of the flight following on after 04:24 UTC as long as there are sufficient position indicators.
In general validation tests should avoid processed ADS-B data and rely on raw ATC or satellite ADS-B data, ACARS or Inmarsat satellite data.
The GDTAAA output from 04:00 UTC to 04:24 UTC is as follows:
https://www.dropbox.com/s/r5a531ju4fk11r9/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200400%20UTC%20A.png?dl=0
https://www.dropbox.com/s/sb2l7rjwkfrryte/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200402%20UTC%20A.png?dl=0
https://www.dropbox.com/s/lrpnjelt2rfot0o/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200404%20UTC%20A.png?dl=0
https://www.dropbox.com/s/e04n6jkqpv35du9/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200406%20UTC%20A.png?dl=0
https://www.dropbox.com/s/poym20w126ykf79/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200408%20UTC%20A.png?dl=0
https://www.dropbox.com/s/fm0jz9zog9av7q5/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200410%20UTC%20A.png?dl=0
https://www.dropbox.com/s/ijcbixaw3lx23w4/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200412%20UTC%20A.png?dl=0
https://www.dropbox.com/s/d726mx6pgfijme5/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200414%20UTC%20A.png?dl=0
https://www.dropbox.com/s/ecxso6hg1xv3aop/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200416%20UTC%20A.png?dl=0
https://www.dropbox.com/s/iyxxy2z6ngga8eq/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200418%20UTC%20A.png?dl=0
https://www.dropbox.com/s/sukxf0kx9pgkgio/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200420%20UTC%20A.png?dl=0
https://www.dropbox.com/s/2d4nfkepa5q2gnl/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200422%20UTC%20A.png?dl=0
https://www.dropbox.com/s/yv0f1d5mbi2431c/GDTAAA%20AMSA%20VH-XNF%2014MAY2021%200424%20UTC%20A.png?dl=0
The WSPR data can be found in the attached Excel table:
https://www.dropbox.com/s/4b02f8rgipdnha9/wsprspots-2021-05-14-0400-0430%20UTC%20Anomalous.xlsx?dl=0
@Richard, @David, all
Uncertainties – Chapter 1 – timing
https://www.dropbox.com/s/grfglmr2u8fj00p/Uncertainties%20-%20Chapter%201%20-%20timing%20.pdf?dl=0
@George G,
Interesting reading!
GDTAA with a 2 minute granularity will never replace ADS-B with more than one message per second. Unfortunately we do not have any ADS-B data from MH370 after 17:21:03 UTC.
In the SIO we only have had up until now hourly BTO and BFO updates. Two minute updates from WSPRnet looks useful, if we can harness it.
@Rob is looking at the waterfalls of the various aircraft detections that he makes. I am looking for data anomalies.
Useful resources are:
https://wsprnet.org/
https://www.voacap.com/hf/
https://wspr.live
http://wspr.rocks
@All,
I am pleased to announce that AMSA have authorised the provision of search and rescue documents in the form of aircraft tasking plans and maps used during the initial MH370 aerial search under a confidentiality agreement and that the ATSB have kindly facilitated their provision.
I am currently analysing the information provided in detail. The general alignment to my validation report above titled “When Whispers Shout” is good, although I am not at liberty to disclose any details.
The photographs supplied by the RNZAF to Brian Anderson under a Freedom of Information initiative align with the new information, as expected.
It is possible that certain media photographs used in some reports, purporting to be from the RNZAF sortie dated 28th March 2014, were in fact stock images of RNZAF taken on other dates during the search. I am investigating which stock photos are correctly dated and timed and will only use such in future reports.
@Richard,
on 26 May 2021 at 13:28 you wrote concerning Sortie #2: “The first aircraft I was tracking appears to be the AMSA Challenger VH-XNC RSCU 440 or alternative aircraft (sortie 2) although it is not in FlightAware.”
Warning, pure speculation follows:
Have you considered it might be the “missing” second Poseidon, there being two mentioned in the AMSA Media release. ?
Perhaps VH-XNC RSCU440, the “local” AMSA Challenger, following having conducted Sortie #1, and now being backed up with the other reserve aircraft (and presumably preliminary flight plans), the “local” aircraft was then on standby for any other contingencies ?
@George,
I did consider that sortie 2 was the other Poseidon. Two things speak against that in my view:
1. The first Poseidon flew from Edinburgh AFB to Learmonth AFB via the search area and did not use the Pearce AFB.
2. The departure for sortie 2 from Perth International Airport fits the arrival from sortie 1 at Perth International Airport plus the usual turn around time.
@All
I am also tracking MH371 on the flight from Beijing to Kuala Lumpur, but here I am using the ACARS position reports sent via the Inmarsat satellite. With the initial flight of MH370 where we have ACARS, ATC ADS-B and radar data we note there is both an issue with accuracy and system latency. The Inmarsat BTO and BFO data is very accurate and there is a small system latency.
Raw ATC ADS-B data for the initial part of the MH370 flight covers a period of 30 minutes 57 seconds. There are 3,586 ADS-B messages in 1,856.82 seconds, a message frequency of almost 2 messages per second. The data is relatively accurate and there is only a small system latency.
ACARS position reports are available for the whole MH371 flight and the initial part of the MH370 flight. The message frequency is every 5 minutes. The data is relatively accurate and there is system latency of less than one second.
Inmarsat BTO and BFO data are available for the whole MH371 and MH370 flight. The message frequency for MH370 after 19:41:03 UTC is every hour. The data is very accurate and there is a small system latency.
WSPR data are available for the whole MH371 and MH370 flight. There are 128,835 WSPR links. The message frequency is every 2 minutes. WSPR transmissions are obviously at the speed of light and there is system latency of less than one second.
@Richard,
Subject: VH-XNF on Sortie #4
Now I have downloaded all of your “GDTAAA output from 04:00 UTC to 04:24 UTC” so I can step through each of the plots, I find I am in a quandary.
Have I lost the plot ?
Please tell me I haven’t.
The last plot, the GDTAAA output for 04:24 UTC, indicates my quandary very clearly. I can see no reason for placing the blue dot hypothetical position where it is other than it is on the line formed by the ADS-B output and fairly evenly spaced along it. It would appear to be nothing more than an extrapolated, or interpolated, data point.
I might or must be missing something.
04:00 – There is an SNR anomaly close, possibly serving as a progress indicator.
04:02 – Like the 04:24 result this seems unrelated to any “Anomaly Links” unless the two, one before and one after were determined to be contributing ? The latitude is midway between the tabled FlightAware values at Central European times 06:01:48 and 06:02:13, so basically on the line.
04:04 – I can’t see the reason that this is a GDTAAA output, other than it follows the line at a sensible spacing.
04:06 – more of the same. Simply seems to be reflecting, or indicating, the ADS output alone.
04:08 –
04:10 – the discrepancy between ADS-B sources/derivations is shown.
04:12 – Possibly a progress indicator, if it can be determined to be so.
04:14 through 04:24
Have I missed the reasoning behind the hypothetical aircraft flight path shown ?
Compare this with a previous plot for the same flight from an earlier posting.
The Post: Richard Godfrey on 18 May 2021 at 18:33, in which you presented:
“At 07:46 UTC another position indicator with 3 WSPR links:”
https://www.dropbox.com/s/2pmxirr8ifs8ft7/GDTAAA%20AMSA%20IN%2014MAY2021%200746.png?dl=0
Regardless of subsequent discussions and revelations concerning this specific data point or GDTAAA position indication it must be noted that this particular data point is only 7.3 Nautical Miles from the actual AMSA aircraft position as indicated by their single frame camera shot “RSCU550 D2S2 POB wide.png”
Source: derived from Google Earth Pro. (I really must learn how to drive that) –
A position “error” of only 7.3 Nm in the GDTAAA estimate is, or will be determined to be is my “guess”, quite reasonable given all the contributing uncertainties.
The data:
Aircraft:
31 deg 14.72 min South, which is 34.245 S
101 deg 50.11 min East, which is 101.835 E
GDTAAA 31.310 S, 101.920 E.
@George G,
You have not lost the plot!
I have failed to communicate properly.
The exercise yesterday using the AMSA Challenger VH-XNF RSCU550 on sortie 4 was not a standard GDTAAA exercise using my standard process. The aircraft departed on 14th May 2021 at 03:23:59 UTC. I analysed the flight path from 04:00 UTC to 04:24 UTC.
Normally I would look for position indicators over a longer period of time and use them to establish the ground speed and track. I would then use progress indicators to confirm the ground speed and track. In the absence of any indicators I would simply predict the position two minutes later based on the previous ground speed and track.
Yesterday I was doing a completely different exercise and did not follow the normal process. I took the last ATC ADS-B data from FlightAware at 04:02:47 UTC and used this data to calculate the position from 04:00 UTC to 04:10 UTC. I took the satellite ADS-B data from “scan Sydney” at 04:19:30 UTC and used this data to calculate the position at 04:10 UTC to 04:24 UTC.
The ATC ADS-B data is subject to a processing delay of up to 2 minutes. The SAT ADS-B data is not delayed.
I analysed the two predictions and they did not match.
There was a latency in the FlightAware processed data of 1.57 minutes.
This was despite the fact that the aircraft initiated a step climb in this timeframe.
I will redo the sequence using the normal process and pretend I do not know either the FlightAware ATC ADS-B data or the “scan Sydney” satellite ADS-B data. The difference will then hopefully become more obvious.
@Richard,
“I will redo the sequence using the normal process and pretend I do not know either the FlightAware ATC ADS-B data or the “scan Sydney” satellite ADS-B data. The difference will then hopefully become more obvious.”
Good.
Thank you for the explanation.
@All
Here is a YouTube video of the progression of the AMSA aerial search for MH370 in 2014:
https://www.youtube.com/watch?v=S98RfOw1Q-U
@All,
My next priorities are as follows:
1. Test and validate GDTAAA based on the AMSA 2021 SAR “D2S2” sortie by VH-XNF call sign RSCU550 from Perth International Airport from the last FlightAware ATC ADS-B detection outbound to the search area and capture of the video and photographic records at the links provided by @David starting at 05:43:50 UTC and at 07:45:53 UTC.
2. Test and validate GDTAAA based on the AMSA 2014 SAR “MH370” sortie by the RNZAF Orion P-3C from Pearce AFB on 29th March 2014 (Day 22). There are 26 photographs (8 interesting) publicly released by New Zealand, which provide GPS date-time stamps and exact location data. There were no SLDMBs dropped during this flight.
This builds on my report titled “When Do Whispers Shout” based on the AMSA 2014 SAR “MH370” sortie by the RNZAF Orion P-3C from Pearce AFB on 28th March 2014 (Day 21). There are 20 photographs (photographs of the “Flaperon” or “White Esky Lid” that are interesting) publicly released by New Zealand, which provide GPS date-time stamps and exact location data. My report erroneously attributed certain media photographs to the sortie dated 28th March 2014, whereas in reality they were taken one day later. There were also 2 SLDMBs dropped during this flight in the search area to mark objects of interest.
It is important to test and validate the GDTAAA system before analysing either the MH371 flight or the initial part of the MH370 flight.
@Richard,
Your progressive plan, per your post “Richard on 28 May 2021 at 15:26”, noted.
As Steps 1 and 2 provide a seven year apart comparison, during your progress through your plan would it be worthwhile to include a onetime visual comparison between the two periods using two graph plots (one in each time period) showing the difference in WSPR link population. This could simply be by taking any two other plots being used in your analysis, but additionally showing all the stable links as well as anomalous links.
Whether or not another step in your plan might end up being tracking of a C130 J Hercules Flight (whose transit flight speed will be similar to that of the Orion) we won’t know until you have taken the prior steps and conducted the analysis.
@RobW, during your Antarctic analyses, and others, have you found, or identified any significant differences between the ability to track, or likelihood of tracking, various aircraft types; for example you did mention the effect of helicopter rotating wings to the best of my recollection.
@George G.
I agree with you that the validation tests for GDTAAA and WSPRnet data need to cover different timeframes, locations and aircraft types.
I obviously have a particular focus on 2014, the southern Indian Ocean and a Boeing 777, to validate the usefulness of GDTAAA in detecting and tracking MH370.
If it can be shown that GDTAAA works for any timeframe, any location and any aircraft type, then it will give a much clearer picture as to the capabilities and limitations of this new technology.
I like your idea of a comparison between 2014 and 2021 in the number of WSPR stable and anomalous links in a particular area.
@All,
A new article by Geoffrey Thomas at Airline Ratings on the help I have received from AMSA and the ATSB with the validation tests regarding GDTAAA and WSPRnet:
https://www.airlineratings.com/news/revolutionary-mh370-tracking-effort-gains-momentum-aussie-help/
@All,
What are friends for?
A friend of mine has a house in the South of France. The house is in a quiet rural area away from busy airports or flight routes.
On Wednesday afternoon 26th May 2021 he was out enjoying the sunny countryside, when his peace and quiet was disturbed by a number of military jets flying overhead on a training sortie. Knowing that I am analysing WSPRnet data to detect and track aircraft he noted the time of the incursion between 15:45 local time (13:45 UTC) until just after 16:00 (14:00 UTC), took a photograph at 16:01:48 local time (14:01:48 UTC) of the contrails and the jets (you have to look close to see any jets in the high resolution picture linked below) and sent me the GPS co-ordinates of where the photograph was taken. The picture was taken looking North-East with an elevation angle around 30°.
https://www.dropbox.com/s/t2n9xl0y5lm83dx/France%20Jets.jpg?dl=0
My friend challenged me to find the jets in the WSPR data!
Linked below is the GDTAAA output between 13:46 UTC and 14:00 UTC during the military jet disturbance in the skies above southern France with the observation point marked in green. The area where the jets were training I have marked with a red circle. The skies above a village called Quézac appears to be near the centre of the training area marked with a red dot. The weather was good with a high pressure zone, little cloud cover and a surface temperature of 17°C. The aircraft appear to be operating at various altitudes from low level up to around 25,000 feet. Contrails usually form between 26,000 feet and 40,000 feet. For this time and location the air temperature was -36.5°C and the relative humidity 100% at an altitude of 25,000 feet, which is ideal for contrail formation. At higher altitudes the wind was dispersing the contrails. In the photograph, you can see there are both some new contrails (which are still not much dispersed) and some older contrails (which are more dispersed). The wind at altitude appeared to be between 30 kmh and 50 kmh from the West (as subsequently confirmed by GDAS data).
WSPR Tx stations are marked with a black dot and WSPR Rx stations are marked with a magenta dot. The anomalous WSPR links at the time are marked in red for a SNR anomaly, orange for a drift anomaly and blue for a dual anomaly. The anomalous short path WSPR links are marked with a bold line and darker red (brown), darker orange (burnt orange) or darker blue. The anomalous long path WSPR links are marked with a bold thin line in red, orange or blue. The midpoint of a short path is marked with a white dot and the midpoint of a long path is marked with a yellow dot. It is possible that the aircraft were mirage jets from the Air Force Base Mont-de-Marsan (base aérienne).
At 13:46 UTC everything is relatively quiet and there is one progress indicator in the area and a number of indicators to the West of the training area.
https://www.dropbox.com/s/boia30astrv5bh3/GDTAAA%20France%2026MAY2021%201346%20UTC.png?dl=0
At 13:48 UTC things start hotting up and there are three progress indicators in the training area, but still a number of indicators to the West of the area.
https://www.dropbox.com/s/c8k78v17faj9vgd/GDTAAA%20France%2026MAY2021%201348%20UTC.png?dl=0
At 13:50 UTC things are still hot and there is a position indicator in the training area.
https://www.dropbox.com/s/fnhvpvcsetn8ivv/GDTAAA%20France%2026MAY2021%201350%20UTC.png?dl=0
At 13:52 UTC things are still hot and there are 2 position indicators in the training area.
https://www.dropbox.com/s/3vmhug5ky54wx6z/GDTAAA%20France%2026MAY2021%201352%20UTC.png?dl=0
At 13:54 UTC things are still hot and there are multiple position and progress indicators in the training area.
https://www.dropbox.com/s/ycx1hwecok65066/GDTAAA%20France%2026MAY2021%201354%20UTC.png?dl=0
At 13:56 UTC things go crazy and there are multiple position and progress indicators in both the training area and the general surrounding area concentrated on various WSPR Tx and Rx stations in the region around Toulouse.
https://www.dropbox.com/s/kjm9r3tsur6uymp/GDTAAA%20France%2026MAY2021%201356%20UTC.png?dl=0
At 13:58 UTC things quieten down and there are two progress indicators in the training area
https://www.dropbox.com/s/jyg28xkoxgytwki/GDTAAA%20France%2026MAY2021%201358%20UTC.png?dl=0
At 14:00 UTC things liven up again and there are three progress indicators and one position indicator in the training area
https://www.dropbox.com/s/lsonsly5inau0tq/GDTAAA%20France%2026MAY2021%201400%20UTC.png?dl=0
A WSPR Tx station F5PRP at JN03po (Dumane, Toulouse, France) sent 61 TxRx WSPR links which were received as anomalous links in this 14 minute timeframe. Anomalous WSPR links were received as far away as Milford, New Hampshire, USA by KM3T at FN42et a distance of 5,708 km.
A WSPR Rx station F5PRP at JN03po (Dumane, Toulouse, Haute Garonne, Occitanie, France) receives 15 anomalous TxRx WSPR links in this 14 minute timeframe from places as far away as Sunderland, North East England (1,275 km).
A WSPR Rx station F4CWN at JN03kn (Saint-André, Gers, Occitanie, France) receives 19 anomalous TxRx WSPR links in this 14 minute timeframe from places as far away as Malmö, Sweden (1,599 km).
A WSPR Rx station F1JSC at JN03ge (Cazaril-Tambourès, Haute Garonne, Occitanie, France) receives 9 anomalous TxRx WSPR links in this 14 minute timeframe from places as far away as Athlone, Ireland (1,287 km) or Bussum, Netherlands (1,068 km).
There are a total of 15 anomalous TxRx WSPR links crossing this general area in this 14 minute timeframe, which have travelled more than 5,000 km. These include both short path and long path TxRx WSPR links. It is interesting to note that the number of short path mid points increases in the general area with the aircraft activity.
The data files used to generate the GDTAAA output are linked below, in case you are interested to follow through with your own analysis:
https://www.dropbox.com/s/u097pjahj0vum0q/France%2026MAY2021%201346%20-%201400%20UTC.xlsx?dl=0
https://www.dropbox.com/s/6t0sz0ksuku504h/GDTAAA%20France%20Flight%20Routes%20and%20Waypoints.xlsx?dl=0
@Richard
Thanks for these report and analyses. This is intriguing indeed to say the least. It looks very interesting and promissing but at the same time it raises some fondamental questions like “along so many thousands km how to be sure that the “tiny” perturbations/anomalies of the signal do not come from multiple objects or simply the local conditions at the reflexion point(s) ? ”
Refering to your remark on the possible impact of aircraft manoeuvering, another question would be “does the aircraft attitude plays a significant role i.e. the interception surface in the path of the signal because of a larger exposure of the wings for example?”
Another point of interest is your remark on the Wake Vortex influence. I am wondering how this would play a role alone as it is just plain air turning in “spiral”. But your remark could make sense to me when considering the wake vortex behaviour when it combines with the hot exhaust air of the engine which is basically always the case after a short time/distance because the physics make the vortices getting closer to each other while descending. At the begining, the different constituants will not mix but turn together. The hot air might have an electromagnetic impact for several physical (density interface?) and chemical composition reasons (Nox ? etc) … to be explored … 🙂
I guess that no signal phase information is available ? It would be of help if it was.
🙂
as we say in French: “Esprit d’escalier” : leaving and getting to the stairway, I had another thought, so I come back :-))
Wake Vortices : it could be simply also the Ice (contrail) captured by the vortices which make a kind of mirror adding a phase shift of the signal…
@Jean-Luc,
In the WSPR community there is a famous story of an amateur radio station call sign UR3RM, a ham radio station in Ukraine who blindly sent out a message on 7.040138 MHz. It was automated. It was text. Maybe someone would hear it. Maybe not!
The “maybe not” part is easy to understand because UR3RM’s transmitter power level was one milliwatt. To put that in perspective, the class 2 Bluetooth transmitter in my iPhone runs at 2.5 milliwatts. The transmission from UR3RM was heard on the island of Tasmania, Australia at a distance of 15,140 km.
Radio transmissions circle the earth at the speed of light around 7.5 times every second. I agree that the signal degrades rapidly bouncing between the ionosphere and the earth’s surface and it may be difficult to receive your own signal after an orbit of the earth.
Prof. Joe Taylor (the inventor of the WSPR protocol) famously stated “there is no need to be loud, as long as you are being copied”. Prof. Joe Taylor received a Nobel prize for physics for his work on pulsars, where he picked up signals that were over 770 light years distant. Closer to home, Prof. Joe Taylor organised a WSPR transmission experiment to bounce signals off the moon and back to earth. If we can detect pulsar signals over huge distances far away in the universe or do Earth-Moon-Earth radio transmissions or pick up a 1 mW transmission from Ukraine in Tasmania, why is it so difficult to believe that we cannot detect “tiny perturbations/anomalies” over so many thousands km?
I am not saying that from a single WSPR anomalous link subject to local propagation conditions, multiple paths or reflected/refracted off multiple objects we can determine anything of value when trying to detect or track an aircraft interfering with those transmission paths. Multiple detections, including position and progress indicators over a period of time, that align with the flight path of an aircraft and fit the predicted position every two minutes based on a known ground speed and track is definitely of value in detecting and tracking an aircraft.
It has been known for some time that the radar cross sectional area of an aircraft depends on the aircraft attitude and when an aircraft turns, climbs or descends then the aircraft “footprint” changes. The aircraft as a physical object is one thing but as you point out the aircraft has a wake vortex. The exhaust gases of an aircraft are not pure air. At the right air temperature and humidity an aircraft will produce contrails. An aircraft wake vortex can cause the density of the air to change or water vaporisation to occur or even ice crystals to form. The propagation of a radio transmission is affected by an aircraft wake vortex over a much larger area than the immediate physical cross sectional area that the aircraft presents.
The WSPRnet database provides certain historical information such as Tx and Rx station identification, location, power level, frequency, SNR or drift, but live WSPR transmissions provide much more information. We are looking at both individual WSPR links frequency, power, location, time of day, season, solar cycle, propagation characteristics, as well as spatial and temporal patterns across multiple links.
https://www.dropbox.com/s/qbxq6xuny0aj8c5/RGsGDTAAA%20Franc%20to1400UTC26MAY2021.png?dl=0
@Gerorge G,
Many thanks for the montage, it really helps to show the timeline.
@Richard. Can you say what numbers of position indicators that look like aircraft there might be that do not lead to or from a continuing track?
To determine they do not, I assume there would be bounds set such as realistic and consistent speeds and the assumption of a straight track, or else there would be a very large number?
Also, a comment re your 29th 1200. If I saw your nine Toulouse diagrams without the red circle I would not pick that area as being noticeably more remarkable for position or progress indicators than others. For example the cocked hat at north by east of the red dot in the below would attract my uninformed attention. What do I miss please?
https://www.dropbox.com/s/yb8obxqfavoyxfl/France%201352%20position%20indicator.jpg?dl=0
@David,
Re your David on 30 May 2021 at 11:20
You bring up the subject I am terming: “Selection and Discrimination”.
This is separate to the other subject: “Uncertainties”.
For both subjects I am (slowly, in the case of the first) preparing some documents, to try to put things into perspective.
If you like they are Reality Checks.
To be fair, Richard did not suggest that he was attempting any tracking in his post on 29 May 2021 at 12:00. He was only presenting the data as he saw it.
The “as he saw it” part concerns his statements such as: “things start hotting up”, “things are still hot”, and worst of all “At 13:56 UTC things go crazy”.
I will call these tendencies of Richards “embellishments”.
Unfortunately, or otherwise, the data does not necessarily reflect the activities of the “number of military jets flying overhead on a training sortie”.
What the data may reflect is a congregation of general activity in many aspects; In other words, congestion.
The data for the eight consecutive two minute time periods may be summarised per the following link:
https://www.dropbox.com/s/k6vtoifftp34ghf/Num%20n%20T.pdf?dl=0
By preferentially scaling two slightly different comparisons I did arrive at the presentation “Chart1”. Whether or not this actually has any true meaning, I am yet to determine. (It helps to leave things for a day or two and then revisit them.). I think unlikely. It is the most I so far have been able to tweeze from the data.
https://www.dropbox.com/s/a3qnkh0fg850px4/Chart1.pdf?dl=0
One of the means I used was to see if there was an increase in SNR for any particular time. I was looking for any indication of a relative increase due to amplification of the WSPR signal due to deflection or reflection by the maneuvering aircraft, such as is seen as a visual glint.
See also second last paragraph of Rob’s post of 31 May 2021 at 15:43 which touches on this or similar subject, but in reference to primary WSPR data.
Well, firstly, I found no result. But, finally, I woke to the fact that what I was doing was NOT going to reveal any difference between times. What I was doing was looking at percentages of the total SNR anomalies for any one two minute time period. WHAT I FORGOT was that for each transmitter/receiver pair (or TxRx) Richard had already been comparing SNR for any particular WSPR Transmission/Reception to the normal-run-of-mill SNR (or average SNR) for that TxRx pair.
However, in any similar, future analysis, comparison of the value of sigma, the number of standard deviations from the average, might assist
_____________________________
Refer https://www.dropbox.com/s/kjm9r3tsur6uymp/GDTAAA%20France%2026MAY2021%201356%20UTC.png?dl=0
There is one fan-out (or handful of spaghetti) from one transmitter near Toulouse Airport. This is 53 “anomalous” transmissions from “A WSPR Tx station F5PRP at JN03po”.
It is considered extremely likely that this is some characteristic of that transmitter at the particular time (i.e. within that particular two minute time period), or is due to some disruption to the transmission by external sources, or even an “eruption” by the transmitter.
Firstly:
Of the 53 JN03po “anomalies”, 38 include “Drift Anomalies” for which the Drift is recorded as either 1 (unity) or -1. For some sort of reality check, this is a 1 Hz per minute deviation (drift of 1 cycle per second over a time of one whole minute) from a transmitter frequency of nominally 14.0971 MHz.
@Rob, @Richard, do I read this correctly ?
Of the 38 JN03po unity Hz/min drift “anomalies”, 30 are Single “Drift Anomalies”.
If only drift greater than 1 Hz/min was considered an anomaly, then would this make the Discrimination between reasonable and unreasonable (anomalous) links more realistic ?
For the record, the resolution of drift in the records is only to the integer. For the 677 anomalous links listed for time 13:56 UTC the recorded drifts vary between -4 and +4.
Of the 38 recorded for JN03po there is only one with size greater than unity, being one result at +2.
Secondly:
The transmitter must be about 6 km from the Toulouse Airport main runway.
If the winds were coming from the North East on that day and aircraft were taking off to the North East, then transmissions, from the “WSPR Tx station F5PRP at JN03po” would likely be affected, particularly in the direction indicated by the handful of spaghetti.
Thirdly,
As radio transmitters, and receivers, may drift in frequency a little, might a transmitter also vary slightly in power output ?
A brief “eruption” in power (my terminology, to suit the purpose) may contribute ?
_____________________________
I had been going to suggest someone take another Wednesday mid-afternoon and do the same analysis Richard has done for 13:46 UTC through 14:00 UTC on Wednesday 26th May 2021.
I expected a similar result.
But Richard has already started an equivalent exercise for a 2 hour period on 26th May 2021 from 13:00 UTC to 15:00 UTC.
I seem a little confused about local Toulouse times, being either one or two hours ahead of UTC.
Perhaps it is a Mirage.
@George G. Like you I am unable to draw specifics from your graph 1 though I follow your endeavours with interest. As to amplitude/SNR you suggest, “However, in any similar, future analysis, comparison of the value of sigma, the number of standard deviations from the average, might assist.” Yes and coupled likewise with the drift rates of changes in dual anomalies, identify not just bank but turning, including in the final descent.
Also, extended duration of anomalies, even if intermittent, across the 2 minutes will give even more confidence in the track of an aircraft being on a continuing course?
My questioning about tracks of other a/c is to help establish what the proportion is of false position and progress indicators – and the reasons for them – so that they may be identified and discarded. All the same, that most probably right now is a low priority refinement.
Still, along these lines I see that in your discussion about that 13:56 Toulouse transmitter burst, implicitly addressed to RobW and Richard, was the comment, “If only drift greater than 1 Hz/min was considered an anomaly, then would this make the “discrimination between reasonable and unreasonable (anomalous) links more realistic?”
Re your “secondly”, a consistent wind would not affect drift change would it?
Re “thirdly”, Richard’s 2 hour comparison has now been completed as of course you know.
@David,
Just a couple of notes before I go to do other things this evening.
“likewise with the drift rates of change” … Good point.
I hadn’t thought about rates of changes in drift being identifiable, but then we do have a range of 9 units to play with, based on what we’ve seen so far.
“including in the final descent” … I don’t really hold out much hope of identifying the descent in March 2014, but that is one aim of what we are trying “here”.
The reason for my less than optimistic view directly above is simply that we only have “secondary data” to play with.
I’m not even sure about the timing of that yet. Is the recorded time when the transmission started or when it is completed and has been completely received, or even, possibly accidentally more ad-hoc. ?
“extended duration of anomalies, even if intermittent, across the 2 minutes” … for this you need “primary” data, such as can be shown “live” on a computer screen. Richard has said that RobW is reviewing what is available, which is mainly 2020/2021 data in respect of the information (flights) we have been discussing, or more descriptively that which RobW has gathered or been able to collect and has analysed.
Noting that I have got to go, briefly:
You write”:
Re your “secondly”, a consistent wind would not affect drift change would it?
I only mentioned the wind, as with a north easterly wind the aircraft take-off from the runway might be towards the north east. If so then warm, even hot, gasses from the engines would be turbulently mixing (or perhaps also mixing otherwise than turbulently) with the much cooler ambient air and would be directly in the path of transmissions from that transmitter.
And direction of aircraft movements from the airport is something that “someone” (not me) could easily check.
Looking at such as the southern France air movements is simply another tool in trying to find the pitfalls, and advantages, of Rob and Richard’s efforts.
@David,
How can I do that ?
NORTH WEST !
Two days apart, firstly on 1 June 2021 at 06:10, and Today on 3 June 2021 at 09:38, I made THE SAME MISTAKE.
Please read NORTH WEST, in lieu of “north-east” in both of those posts and any other associated comments I may have made recently in which I erroneously described the direction of interest as “north east”.
@All,
My friend with the house in France asked is this a “possibility or a proof?”
My answer to him is that this is only a “Possibility”.
“Proof” in my view would be something like:
1. 10,000 15 minute slots without any aircraft.
2. 10,000 15 minute slots with aircraft.
A high correlation with aircraft and a low correlation without aircraft.
High resolution video evidence of each 15 minutes slot.
Precise times and GPS data for (a) the camera position and (b) the extent of the aircraft training area.
Precise information giving the type of each aircraft and the aircraft’s ground speed, track, altitude, contrails and contrail dispersion.
Precise information giving the air temperature, wind speed and wind direction both on the surface and at each relevant aircraft altitude.
Unfortunately this is not realistic in the short or even medium term.
My friend also was clever enough to take a picture of the exact time from the Paris Observatory web site.
His picture on 26th May was probably taken at a time closer to 16:58:27 Local Time.
We compared the EXIF data with the Paris Observatory data in the following picture:
https://www.dropbox.com/s/qwkhpbn8nuhk72s/Paris%20Observatory%20Time.jpg?dl=0
For a repeat of this exercise at some point in the future, my proposal to my friend is:
1. Take a picture when the military aircraft first appear in the area.
2. Take a picture from the same observation point when the military aircraft last appear in the area.
3. Take a picture from the same observation point 20 minutes after the last military aircraft disappeared from the area and there were no aircraft in the area for that 20 minutes.
4. Take a picture of the Paris Observatory web site time just after picture 3.
5. Note the GPS co-ordinates of the observation point (latitude, longitude, elevation).
6. Note the approximate extent left and right, up and down of the aircraft from the observation point (e.g. left at 330°, right at 060°, up at horizon + 30°, down at horizon +10°).
The iPhone has a Compass App with an extent function (tap once to initiate, rotate to get extent, tap again to clear) and a Measure App with and elevation function (tap level to initiate).
7. Note the surface temperature, wind speed and wind direction from either as °M ot °T. (I can get this from GDAS as well).
8. Note any contrails and estimate the dispersion rate (e.g. thin line at 14:10 local time, fat line at 14:20 local time).
9. Note any cloud formations (cloud cover), layers (low or strato-, middle or alto-, high or cirro-, multi-level) and cloud type (discrete lumps or cumulus/altocumulus/stratocumulus, wide layer or stratus/stratocumulus/altostratus/cirrostratus/cirrus, tall multi-level or nimbostratus/cumulonimbus) and cloud colour (white, grey, dark grey) (e.g. your picture on 26th May 2021 shows 10% cover, low, cumulus or stratocumulus, white).
https://www.dropbox.com/s/g27xbl2blgkpl0r/Cloud%20Types.png?dl=0
10. Use binoculars to see if the aircraft type can be detected.
11. Note the direction the aircraft appear from and disappear to.
@All,
A comment from Ed Anderson and my reply at AirlinerRatings:
https://www.airlineratings.com/news/revolutionary-mh370-tracking-effort-gains-momentum-aussie-help/
I don’t know if it helps, but living close to Frankfurt Airport, I observed an unusual go-around of Lufthansa D-ABYA. Later I found out that the aircraft had been parked in Hamburg due to Corona and had been flown back to Frankfurt for further service on June 3,2021. Touchdown in FRA was 12:08 UTC, the whole flight track is a available on flightradar24.
It is interesting from my perspective and thus might be helpful to testing your new technology, because of the unusual flight path, going from Hamburg to Rügen, then flying a circle near Leipzig and finally doing a go around over the city of Frankfurt, while a typical go around would usually just involve a straight climb and a much later turn to the general approach routes.
Glad if that could be of any help
Tom
Concerning:
Richard on 29 May 2021 at 12:00 to @All,
Subject: What are friends for?
I have taken the Excel File “France 26MAY2021 1346 – 1400 UTC”
and started on some summations and calculating percentages.
The following graph shows the numbers of Anomalous Links listed in the file, SNR, Drift and Double.
Where there is a Double Anomaly listed in the file, the numbers shown in the graph counting as a Double are contributing to BOTH the SNR and Drift counts as well.
I have calculated percentages of Total Anomalies and percentages of Total Links as shown by the spreadsheet.
Excel and I have had one of our frequent disagreements, so tonight might not be the time, but anyone is welcome to it. Ye only have to say.
https://www.dropbox.com/s/zc8othy2jaocxz1/Number.pdf?dl=0
If the graph has faults, then refer to above mentioned dis-agreement.
@George G,
Excellent! Many thanks for the Excel spreadsheet.
You idea got me thinking and I have produced a larger set of source data over a 2 hour period on 26th May 2021 from 13:00 UTC to 15:00 UTC.
This covers the period where the french military jets were in the vicinity of my friend’s house, but also before and after. I have included the stable as well as the anomalous links.
There are around 2,600 WSPR links every 2 minutes. 15% have an SNR anomaly, 19% have a drift anomaly. 32% have a single anomaly (either SNR or Drift) and 2.6% have a dual anomaly (both SNR and Drift). Standard deviations are reasonable over the 2 hour period.
The file is 24 MB and contains 159,918 rows.
https://www.dropbox.com/s/2qscop4ztpkn4eq/wsprspots-2021-05-26%20France%201300%20UTC%20to%201500%20UTC.xlsx?dl=0
@Richard,
In your post above on May 2021 at 21:04, you say “May 2021 at 21:04”.
Presumably that was thanks in advance:
https://www.dropbox.com/s/fkr2cyh1bapvmzb/Fr%20WorkFile%20re%2026MAY2021%20%20-%20pm01JunCopy.xlsx?dl=0
That should have read:
In your post above on May 2021 at 21:04, you say “Many thanks for the Excel spreadsheet”.
Presumably that was thanks in advance.
@George G,
Very witty!
@All
While preparing a list for Donato to get started with his thesis it came to my mind that such a starting list might be interesting for anybody who wants to get started on this subject regarding MH370 and WSPR:
I have gathered many data, notes and references over the last months from Mid 2020, anyway not in one small file but distributed on a couple of notebooks.
Numerous things ( incomplete list) might be interesting for you to get started:
Database:
http://www.wsprnet.org that contains the original secondary WSPR data in March 2014
http://www.reversebeaconnetwork.net that contains Skimmer data on VK6 signals in CW during the flight time of MH370 to check propagation conditions during flight duration.
http://www.wsprnet.org that contains the original secondary WSPR data of the AMSA SAR missions in March 2014
http://www.wsprnet.org that contains the data of flight QFA114 on 13.November 2020 from JNB (Johannesburg, South Africa to PER, Perth in Western Australia) passing close to the IG LEP at app. 22:48 h UTC in the SIO. This was just one of the rare repatriation flights during COVID-19 in 2020 between South Africa and Western Australia.
http://www.wsprnet.org contains the original secondary WSPR data of the AMSA rescue mission west of Australia on 13. May 2021 to 15. May 2021 for the Indonesian fishing vessel in distress
https://vt.superdarn.org for SuperDARN radar data in early March 2014 of Chinese SuperDARN Radar ZHO (beam 0 towards Cape Leeuwin in Australia) in Antarctica before and after midnight 07./08. March 2014
Stored screenshots of primary WSPR data (www.kiwisdr.com) from various aircraft as pointed out in the ePaper for the iSession at Hamsci2021.org in March 2021
WSPR data in Dropbox
ADS-B data:
Flightradar 24 (FR24) for the position data of aircraft
Flightaware.com for the position data of aircraft
https://mh370.radiantphysics.com for position data MH370 til 17:20 h UTC and other useful information
http://www.mh370search.com for various position data MH370 and other useful information
several videos on Youtube for FR24 ADS-B data until app. 17:20 h UTC, no more MH370 data after but for other aircraft
http://mh370-captio.net CAPTIO website and CAPTIO video on Route N571 “Traffic on N571 around 18:40 h UTC” showing MH370 and other aircraft
References
Numerous articles on WSPR, RCS data on HF for ISS (50dBsm = 100.000 sq.m. at 30 MHz) and aircraft, HF backscatter in the Southern hemisphere from DST Group et al.etc.
My Spreadsheet, notes and comments on flights MH370, QFA114 and others
My Screenshots on various KiwiSDR WSPR signals in Antarctica from various aircraft heading towards Antarctica from South America (Southern tip), South Africa (CPT), Australia, especially Hobart in Tasmania (HBA), from Newzealand (CHC: Christchurch) and many Qantas joy flights QFA290X from ADL, BNE, MEL, PER, SYD towards Antarctica and retour.
http://www.spaceweather.com for space weather at date and time of observation, also http://www.kc2g.com
http://www.airscout.eu for airscatter mainly on VHF/UHF and higher frequencies
Live KiwiSDRs: Monitoring live WSPR signals on KiwiSDRs, i.e. ZL1PWM til 31.01.2021 (CL after that date) and others.
Monitoring flights SIA478,SIA479 between Johannesburg (JNB) and Singapur (SIN) and retour with WSPR primary and live signals from KiwiSDRs in South Africa (ZS), Réunion (FR, F61695), Indonesia (SWLOI33), Philippines (DU6/PE1NSQ), Hawaii (AI6VN/KH6) mostly on 7 MHz ad 10 MHz
I follow a 3-step process called OVV, such as observation – validation- verification that can comprise
Live primary WSPR KiwiSDR data (waterfall) if available (www.kiwisdr.com)
ADS-B data from FR24 and/or flightaware.com or others
Extraction of secondary WSPR data from http://www.wsprnet.org during flight time
SW VOACAP Ham Online Software for propagation path and probability of circuit
SW Proppy (occassionally)
SW Proplab 3.1 (for hop calculation if required)
In general I observed aircraft movements in Antarctica as there are not too many down there to avoid confusion. I choose DP0GVN as TX and ZL2005SWL near Nelson on the Northern coast of the Southern island of Newzealand. The regular propagation path is over Antarctica and close to the South Pole (181 degrees) similar to the
assumed heading of MH370 (180 degrees).
In general there was good propagation during Antarctic summer (October 2020 to March 2021), now during Antartic winter propagation is not good and no flights..
March 2014 was close to the peak of solar cycle 24 in April 2014 so there were excellent conditions for propagation even on 10 m (28 MHz).
In order to get started I suggest choosing some KiwiSDR in WSPR mode from http://www.kiwisdr.com around busy airports and study the impact on primary WSPR data between various WSPR stations and others during straight heading and intense bank-to-turns during approach, loitering and/or departure of aircraft.
Interesting things will happen to the WSPR signals, some will not be decoded and one will not find these spots in the WSPR database for that very reason. Extend range when more famaliar with procedures. I hope this will help to get started! Thanks and good luck!
@BG370. Not about WSPR but the new pics of MH370 flotsam you forwarded to Geoffrey Thomas are below. Nice head levelling with the girls, photo 8:
https://www.airlineratings.com/news/new-images-mh370-debris-revealed-wreck-hunter/
@Rob,
Your Rob on 31 May 2021 at 15:43 – Good Work.
Like your suggestion concerning using busy airports to study effects.
Also, areas containing busy airports will often be areas of congestion which follows on from part of my separate reply to David on 30 May 2021 at 11:20.
@Richard,
Your Richard on 31 May 2021 at 21:04 – Your look at a larger set of source data is similar to what I was going to suggest may be of assistance (even if now only half heartedly as you may have effectively done so) in my separate reply to David on 30 May 2021 at 11:20, concerning your France subject.
I can now complete and send that reply knowing for sure that you will by now have already seen David’s post. (And …. that completion has taken SOME time).
And, Richard, thank you for your earlier short list of sites of 27 May 2021 at 15:25, (which Rob has now greatly extended above, with other suggestions and hints).
From that, my visits to http://wspr.live and others more recent to a couple of ham sites they seem to indicate that transmitter and receiver latitudes and longitudes may be available for all. Rob, Richard, am I correct ? This may reduce positional uncertainty somewhat.
Also, does or does not the wspr.live database mentioned as “The heart of this system is the database service containing all the spots ever reported to wsprnet.org” actually include latitudes and longitudes for all the transmitters and receivers likely to be in operation in March 2014. ?
@George G,
The latitude and longitude in WSPR Live is derived from the Maidenhead Grid code in the same way as I do in GDTAAA. It is no different.
Ta
@David
Thank you for posting this article. The local people in Madagascar have been much more welcoming and helpful in my debris searching efforts than many keyboard warriors on the internet. Local people have found and brought me more 370 debris than I have been able to find myself.
@George G,
As you have noted, I have now expanded the data set that I published to include the time before and after the training exercise. If you do not have a with and without comparison, the graph can be misleading.
I am calculating all the global WSPR links and the percentages of stable, SNR, drift and dual anomalies at each point in time. If you do not know what percentage of links are stable or anomalous, the graph can be misleading.
I want to next work out how many of the global links actually cross the general area of interest as well as actually cross a target area (as marked with a red circle) within a general area of interest.
I then want to work out how many of those links intersect in the area of interest or target area.
I plan to capture these statistics for any scenario analysed. Since in this example, there are 159,918 links in the timeframe, I may need to buy a supercomputer.
This is a frequentist approach, which I plan in addition to my pattern recognition approach. I am preparing a longer comment on pattern recognition.
@David
My apologies for the delay in replying to your excellent questions.
You asked: “Can you say what numbers of position indicators that look like aircraft there might be that do not lead to or from a continuing track?” I have not kept a statistic, but in general there are more potential aircraft tracks that come to nothing, than the few tracks that fit the position indicators and fit a plausible ground speed and track. I went into the probabilities a little in my first paper (numbers of commercial passenger and cargo aircraft, commercial aircraft cruise speed, flight route and flight plans, separation rules, number of aircraft not in ADS-B data, private aircraft, military aircraft, etc.) Of course, the french jets would not keep to separation rules. Generally speaking, commercial aircraft fly at very predictable speeds along designated flight routes with a clear separation. MH370 is an exception. The pilot in command will be concerned about flight routes, fuel savings and separation from the point of view of getting away from anywhere rather than getting to somewhere.
I have not tracked the french jets. This was just a frequentist look and see if there was any WSPR data support for the fact that there were a bunch of jets in a particular ill defined area. This was not a detailed look to track each jet individually and see what their ground speed was and what track they followed. Given their turns, manoeuvres, climbs, descents, etc. tracking a military jet will be difficult.
I have now looked more closely at the position indicator you indicated with the black arrow. There are 5 anomalous WSPR links intersecting at the same location at 13:52 UTC:
https://www.dropbox.com/s/zt12uzucafs4tfa/GDTAAA%20France%2026MAY2021%201352%20UTC%20Local%20Aircraft.png?dl=0
Looking on a global scale at the same position indicator, it comprises 5 long path links including 2 drift anomalies (JN53fuKM56vo and DN40hmDM16xf) and 3 SNR anomalies (JO47wdJP94at, KO30coJN47lh and EM82emFM16qv):
https://www.dropbox.com/s/m1gm2e4sgvtfyqu/GDTAAA%20France%2026MAY2021%201352%20UTC%20Global%20Aircraft.png?dl=0
Assuming an aircraft has been detected by this position indicator, I searched ADS-B data for that location and time. There were a number of commercial jets passing that location EJU7594 ZRH-OPO, JAF3682 TLN-BES, AFR6132 ORY-TLS but they kept to the recognised flight routes which are at least 20 nm distant and did pass at that time. There were a few private aircraft passing that location, F-JHEE from Ville-Franche (LFHV), F-BNCQ from Rodez (LFCR) and F-GNZY from Limoges (LFBL) but again not at that time. The rescue helicopter SAMU15 based at Aurillac Airport (LFLW) flew 2 sorties that day, one local sortie at 10:06 local time and another to Clermont Ferrand at 19:22 local time, so that did not fit.
I was intrigued that I could not find the aircraft in the ADS-B data, so I decided to track the aircraft. At 13:54 UTC there was another position indicator with 3 anomalous WSPR links intersecting:
https://www.dropbox.com/s/8xm1tfezuutfi04/GDTAAA%20France%2026MAY2021%201354%20UTC%20Local%20Aircraft.png?dl=0
Again, looking on a global scale at the same position indicator, it comprises 2 long path links including 1 SNR anomaly and 1 dual anomaly (IN80dkIL38bo and FN42ihEM97kj) and one short path link, which was a drift anomaly (JM19iqJN18ar):
https://www.dropbox.com/s/qsnbn11artvyei6/GDTAAA%20France%2026MAY2021%201354%20UTC%20Global%20Aircraft.png?dl=0
I was surprised that the aircraft had not moved very much in the two minutes interval. Was it a helicopter hovering around, perhaps the French Air Force always have a helicopter on standby in the area when they are doing a training exercise, in case a pilot has to eject. I did not see why a helicopter would not put down in a field somewhere and wait to be called if required.
I was even more intrigued and continued to track the aircraft for 40 minutes through a total of 14 position and 6 progress indicators. The track included periods of circling at a particular location followed by moves to a new location. The circles seemed quite tight and movement to a new location was only at a ground speed between 80 knots and 100 knots. The track of the aircraft is shown in the following link:
https://www.dropbox.com/s/oub2qmeh4mnjqdv/GDTAAA%20France%2026MAY2021%201434%20UTC%20Local%20Aircraft.png?dl=0
The aircraft was a glider and probably a high performance sail plane from a local gliding club. The gliding weather report predicted excellent conditions that day. What surprised me is that this area was not categorised as air space where a transponder is obligatory, as there is a small airport at Aurillac and the area is used for military aircraft exercises. Maybe I did not look hard enough in the ADS-B data to find this glider. Alternatively, I could track the glider until it lands somewhere and find out at which airfield and give them a call.
I now understand why @Rob prefers to use the Antarctic for his research, there is less aircraft traffic and it is easier to identify a detection.
You also asked: “What do I miss please?” The answer is you don’t miss much. Your questions are always “bonzer!”
@Richard. “……but in general there are more potential aircraft tracks that come to nothing, than the few tracks that fit the position indicators and fit a plausible ground speed and track”. Thank you.
As to whether apparent tracks can be coincidences in a mass of anomalies, as I understand it that is essentially ruled out by such as the discriminating points that you mention. There can be the obverse though, when anomalies result from bank or momentary conditions, in which case there might be just the one; or they could be very intermittent, so that their continuity does not become apparent.
So lack of a track does not mean necessarily that an anomaly was not created by an aircraft.
I see from your posts to George G., 1st June, 06:25 and 3rd June, 07:41 that you are, “preparing a longer comment on pattern recognition” and are working on your “frequentist approach”. Great! These should help.
Some comments now about your serendipitous finding of a track from that north-by-east position indicator and conclusion that it was a glider from the track’s characteristics and elimination of other possibilities.
What a surprise that a glider would produce such anomalies, with its limited vortices, cross section and lack of metal, even though they have the wingspan.
Also that you get an anomaly each 2 minutes, yet with one or two apparently straight penetrating courses longer than that, ie the anomaly not just being the result of a brief bank on the ‘right’ heading.
I estimate that over the straight mid-section 4 mins he was making 120 knots ground speed-of-advance which might need wind behind and/or could be the result of approximations.
A mild surprise is that the Toulouse and other local stations did not ‘get a guernsey’ amongst those transmitting and receiving anomalies.
Unfortunately were you to track it with the aim of confirming that a glider, I imagine the track would peter out as he descended, so most likely its landing site, which also might have been his launch site, might not be located by WSPR. Possibly the same for MH370.
That said, past detections at long range suggest low level detections are possible, in some conditions anyway.
Thanks too for the “bonzer” adjective. Australian colloquialisms are beset by the ubiquitous TV and yet expanding social media.
Internationally we are all becoming homogenised – for both good and ill. Culturally that is: politically not quite.
@David,
In 2021 there are a large number of WSPR links, but in 2014 there are only a few by comparison.
It may be that we need to set the threshold higher for detecting a SNR or drift anomaly when there are so many WSPR links. For example, 2 standard deviations for a SNR anomaly and 2 Hz/minute for a drift anomaly. This would reduce the clutter in the GDTAAA output and help to see the wood for the trees.
@Richard. Yes, George G suggested lifting the 1Hz drift anomaly and extending that to SNR also looks like a good move. Maybe just when there are so many WSPR links as you say, and for now.
@Richard, all
No drift 1 2 3 4 >4 TOTAL
129282 19649 7152 2609 1226 0 159918 Samples
0.8084 0.1229 0.0447 0.0163 0.0077 0 100%
Until I press “Submit” I won’t know if formatting is retained.
@Richard,
“NumT” derived from data from your database “wsprspots-2021-05-26 France 1300 UTC to 1500 UTC” posted on 29 May 2021 at 12:00.
Note that this shows “raw” number data not percentages.
https://www.dropbox.com/s/h0ilcqbsr1tpryt/NumT.pdf?dl=0
@George G,
Many thanks for your graph of the raw numbers of the Global WSPR links and the anomalies on 26th May 2021 between 13:00 UTC and 15:00 UTC.
Here are the graphs for the percentages of SNR anomalies and percentages of Drift anomalies in the same timeframe.
https://www.dropbox.com/s/6wwloeuzaml4ja6/GDTAAA%2026MAY2021%201300%20to%201500%20UTC%20%25%20SNR%20Anomalies.png?dl=0
https://www.dropbox.com/s/eep82md00k40r3b/GDTAAA%2026MAY2021%201300%20to%201500%20UTC%20%25%20Drift%20Anomalies.png?dl=0
Do you agree that the global frequentist view does not show any evidence that something was going on locally in the south of France?
What I need to do is filter the global numbers to include only those WSPR links and percentage of SNR and Drift anomalies that crossed the area of interest in the south of France and see what the percentages of anomalies are by comparison to the global view.
@David,
The position indicator at 13:52 UTC that you pointed out in the french military jet scenario stood out to you even amongst the many other anomalies in a chart covering a wide area:
https://www.dropbox.com/s/3vmhug5ky54wx6z/GDTAAA%20France%2026MAY2021%201352%20UTC.png?dl=0
You are looking at an area of 81,667 sq nm in the above chart.
Obviously it helps to clear out the other anomalies and only look at the 5 intersecting anomalies at the point of interest:
https://www.dropbox.com/s/zt12uzucafs4tfa/GDTAAA%20France%2026MAY2021%201352%20UTC%20Local%20Aircraft.png?dl=0
What helps even more is to change the scale and when you look more closely you even find that it is not 5 intersecting anomalies, but probably only 4:
https://www.dropbox.com/s/i21ik62etzmuq4l/GDTAAA%20France%2026MAY2021%201352%20UTC%20Zoom.png?dl=0
I tracked the glider for 42 minutes, every 2 minutes, which produces 21 legs along a track with a set of 22 positions. There were 14 position indicators, 6 progress indicators and 2 positions unsupported by any indicator. In the absence of any indicator I assume the track continues at the previous instantaneous ground speed and previous instantaneous track.
The inherent lack of precision of around 18 nm means that it does not make sense to report the ground speed over a single leg, but I can assure you in none of the legs the ground speed was 120 knots. I can give you all the data, if you are interested to dig deeper.
The 4 position indicators with 3 or more intersecting WSPR anomalous links stood out from all the other links around. Out of the 10 position indicators with only 2 intersecting links, 7 out of 10 stood out and were clearly separate from any other candidates in the vicinity of the glider’s predicted position. These other candidates were not reachable on any track at any reasonable ground speed. A high performance glider can only handle a maximum air speed of 150 knots, but most of the time will be between 50 knots and 100 knots.
I report average ground speed over all legs since the last position indicator, but with a minimum of at least 5 legs being taken into consideration. I report initial bearing and final bearing over each leg. This is checked for consistency and can be slightly corrected when a new position indicator is reached. The average ground speed of the glider over 42 minutes was 34.5 knots.
Obviously to track a glider was more challenging than say an AMSA SAR jet in transit from Perth to the search area. The glider is circling and soaring, whereas an AMSA SAR jet is in straight and level flight apart from any step climbs. The track of the glider is not a question of serendipity, but a considerable and detailed analysis. I was also surprised that the initial position indicator led to this aircraft type, principally because I could not find any ADS-B data in this case. There are many other gliders in this area which can be tracked on ADS-B data.
@David,
In the link below I show the track of the glider in the gentle wind from the West as the pilot follows the line of the hills. The highest point in the area where my friend lives is 798m but the Massif Central rises up to 1,886m. When the wind from the ocean crosses the plain and then hits the hills it provides uplift for a glider:
https://www.dropbox.com/s/ctcq0z200d8nuzu/GDTAAA%20France%2026MAY2021%201434%20UTC%20Relief.png?dl=0
Nice
@Richard. Thanks for your assurance that in none of the legs was the ground speed 120 knots and likewise for your advice about that ridge lift.
“I can give you all the data, if you are interested to dig deeper”. Thank you but no need.
My thanks too for detailing aspects of your methodology, which helps understanding.
Well can I see that it would be demanding to track this while rejecting infeasible position indicators, deducing the aircraft type in the process.
@Richard,
It may be akin to taking the low hanging fruit first, and then seeing how to easily get the rest.
Your 3 June 2021 at 07:41, above, and also your 3 June 2021 at 10:16 to David lead the subject towards the problem of selection and filtering. As you know I have my sights on shooting down the use of unity Hz/minute as an indication for a drift anomaly, and I am gathering evidence or ammunition towards this. It may be that its use has no effect one way or the other on some results. To be confirmed. But that is certainly not conclusive at this stage and based on a small set of data. Nor may that, alone, be sufficient reason to reject its use.
I will start to prepare a comparison between retention or rejection of the unity drift value for the two hour period in the south of France.
I agree, that so far there is not enough light yet with the France data set to see the wood through the trees, and quite possibly there never will be.
But I do not simply agree that “we need to set the threshold higher … … when there are so many WSPR links”.
Perhaps a standardised approach to progressively increasing, or decreasing, selectivity of the data weeding process (or fruit harvesting) should be taken.
At each stage of this process “a look through the trees” should made to see what might be there.
On a slightly different subject in your reply of 1 June 2021 at 10:00 to David, the two “global” presentations “… 1352 UTC Global …” and ” … 1354 UTC Global …” graphically highlight how two close (Tx and Rx) radios on one side of the world might result in large position uncertainties on for where “long paths” pass by on the other side of the world. (Richard, I know you are not including very close TX/RX pairs just for this reason.
The above was written very late last night and saved. I have now re-read, tidied second paragraph, completed the last paragraph, and decided suitable to post.
@All,
Continuing the validation test with the AMSA Challenger SAR jet (registration VH-XNF and call sign RSCU550) en-route to the search area to help the Indonesian fishing vessel D2S2 (Sortie 4), here is the output from GDTAAA from around the last ADS-B position where the aircraft goes out of range of the ATC ADS-B station from 04:00 UTC onwards:
https://www.dropbox.com/s/kkihbwb0kyahwvy/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200400%20UTC.png?dl=0
https://www.dropbox.com/s/hp7tlyt1hsbowax/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200402%20UTC.png?dl=0
https://www.dropbox.com/s/ey59glf1k4secpb/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200404%20UTC.png?dl=0
https://www.dropbox.com/s/aiglfp7ybe9gdbe/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200406%20UTC.png?dl=0
https://www.dropbox.com/s/i7j5am0l0il4w5v/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200408%20UTC.png?dl=0
https://www.dropbox.com/s/5aw71qnl16ns1im/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200410%20UTC.png?dl=0
https://www.dropbox.com/s/v7rxti91eoks04s/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200412%20UTC.png?dl=0
https://www.dropbox.com/s/1v5tfjirs1f7et7/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200414%20UTC.png?dl=0
https://www.dropbox.com/s/yrgr361g9ieeyqs/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200416%20UTC.png?dl=0
https://www.dropbox.com/s/p9rd5l5qrnuib4p/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200418%20UTC.png?dl=0
https://www.dropbox.com/s/v121bkrsbjcg2qr/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200420%20UTC.png?dl=0
@George G, @David,
I plan to complete the validation test based on the AMSA Challenger (registration VH-XNF and call sign RSCU550) Sortie 4 without making any changes to the SNR or Drift limits.
For the next scenario the AMSA RNZAF MH370 Sortie on 29th March 2014 I will also keep the same SNR and Drift limits. Firstly the flight took place in 2014 with fewer WSPR links and secondly I want to be able to compare the results with a similar flight by the same aircraft on the day before.
What might be a useful exercise is repeating a 2021 scenario, where we have results already but with setting the SNR limit to two standard deviations and the Drift limit to at least ±2 Hz/minute.
@Richard
1. VH-XNF
2. RNZAF (P3)
3. Repeat run with higher selectivity
All three sound Good.
Question: How much work and time would it be/take for you to automate obtaining two (or more) sets of results during a “run” where each of the sets has been generated using a different selectivity setting ?
@George G,
You asked: “How much work and time would it be/take for you to automate obtaining two (or more) sets of results during a “run” where each of the sets has been generated using a different selectivity setting ?”
There are 3 stage in the process:
1. Data Preparation.
1.1 Download WSPRnet data for the timeframe of the flight.
1.2 Remove Tx and Rx stations with only 4 character Maidenhead Grid reference.
1.3 Remove WSPR links that are within the same 2 character Maidenhead Grid.
1.4 Set the Time UTC from the Unix Timestamp.
1.5 Set the 12 character WSPR link.
2. Limit Setting.
2.1 Calculate the mean and standard deviation transmission distance adjusted SNR for each transmission power level.
2.2 Set the SNR anomalies.
2.3 Set the Drift anomalies.
3. Output Generation.
3.1 Set the relevant waypoints and flight routes.
3.2 Set the starting point and initial ground speed and track.
3.3 Predict where the aircraft will be in the next two minutes.
3.4 Generate the output for the next 2 minute step.
3.5 Analyse the output and if required adjust the ground speed and track.
3.6 Repeat from 3.3.
Most of the process is automated, but the analysis in 3.5 is time consuming.
Data preparation is time consuming for flights in 2021, but easy for flights in 2014.
For example:
MH371 (7th March 2014) was 5h 54m duration and there are 36,940 WSPR links.
RNZAF (28th March 2014) was 9h 34m duration and there are 74,501 WSPR links.
RNZAF (29th March 2014) was 9h 54m duration and there are 76,795 WSPR links.
VH-XNF Sortie 4 (14th May 2021) 6h 23m duration and there are 780,624 WSPR links.
2021 datasets have to be cut into pieces of about 1 hour each.
Changing the SNR or Drift limits is easy.
An 8 hour flight will generate 241 GDTAAA outputs, this is time consuming both to analyse and report.
@George G, @David,
There are ways to simplify the process.
WSPR currently uses 17 Bands. The Band of operation is computed from the frequency as an index for faster retrieval, as an integer representing the MHz component of the frequency with a special case for LF (-1: LF, 0: MF, 1: HF 160m, 3: HF 80m, 5: HF 60m, 7: HF 40m, 10: HF 30m, 14: HF 20m, 18: HF 17m, 21: HF 15m, 24: HF 12m, 28: HF 10m, 50: VHF 6m, 70: VHF 4m, 144: VHF 2m, 432: VHF 70cm, 1296: VHF 23cm).
Power is as reported by the transmitting station to the receiving station during the transmission. Units are dBm (decibels relative to 1 milliwatt; 30dBm=1W). Typical values are 0-50dBm, though a few are negative (< 1 mW). The 20 current dBm power levels used in WSPR (0, 3, 7, 10, 13, 17, 20, 23, 27, 30, 33, 37, 40, 43, 47, 50, 53, 57, 60, 73).
83% of WSPR stations use 4 Bands (3 MHz, 7 MHz, 10 MHz, 14 MHz) and 6 Power levels (20 dBm, 23 dBm, 27 dBm, 30 dBm, 33 dBm).
86% of the WSPR anomalies are covered by these 4 Bands and 6 Power levels.
At this validation stage of GDTAAA, I prefer to use all Bands and all Power levels. It might be interesting to run a test just using WSPR links where the Tx stations transmit at say, a Power level of 23 dBm and on the 7 MHz Band (11.5%).
The Bands and Power levels will change depending on the propagation conditions. The above figures are based on the WSPR data from 13th May 2021 between 20:00 UTC and 24:00 UTC.
My view at the moment is that we need all the data we can get.
@Richard said: “My view at the moment is that we need all the data we can get.”
1. That makes sense. (See, Paragraph 3 below)
2. Being able to see through different filters for comparison is informative, whilst having the data at hand.
3. It is understood that when one begins to weed out any data in an analysis, there needs to be an argument for why that particular data is considered no longer relevant to the analysis.
4. That you found you had tracked a glider was quite an eye opener. It may be of interest to find from what material that aircraft was made, and what frequency bands were being affected by it. (I’m no way suggesting you divert your attention to it, but someone else might.)
5. As you track VH-ZNF, RCU550, I’m interested in what of SNR, Drift and Double Anomalies actually become of most use. In particular, whether or not Drift is a significant contributor to such a flight tracking. I’ll attempt to keep track.
@Richard,
Having time now to digest the immensity of the task you have described:
You wrote: “An 8 hour flight will generate 241 GDTAAA outputs”,
for which it is time consuming to firstly analyse, then report.
Yet, this may seem the only method, or the method most likely, to reveal any potential pitfalls.
For example:
Selectivity:
Should Selection of items of interest, being potential “Position Indicators” and/or “Progress Indicators” be misguided, or
Discrimination:
Should the method of Discrimination for those Selected items of interest between:
1: items (or Indicators) which may actually be resultants from the flight being tracked, or
2: items which are much more likely to be due to other (external) factors,
be similarly misguided,
THEN: It is time during validation exercises to find need for improvements, and subsequently make improvements.
For example, if a track not actually representing the true flight were to be followed then it is preferable that this be one in 2021, no the final flight of interest of 2014.
If, say, a track was followed which started out being the aircraft being tracked, but misguidances resulted in unknowingly changing over to the track of another aircraft and then subsequently tracking that other aircraft, the time of validation is the time to find out.
Similarly, if a track started out reasonably correctly, but there were indications leading off the track, then this possibility should also be rigorously searched and probability of it occurring minimised. The time of validation is the time.
What could be VERY instructive, if an aircraft was being tracked and the tracking was led of the real track, due to such as “congestion”, would the track “run out of steam” or peter out.
Richard,
I’m not here suggesting you haven’t considered, or come across, these factors or potential problems already and developed some solutions. What I’m trying to do is list them out in the open, and also develop some framework around what I term “Selection and Discrimination”. For discussion as time goes on.
@George G,
My general communication policy is openness and full transparency. I believe that I have shared all my data, process and results in full.
I appreciate there may be some issues that only interest one individual or a question that is off the mainstream discussion, which are more appropriate for a private discussion. I welcome private emails and receive a large number and answer them all.
I appreciate and welcome that you are trying to develop a framework for discussion on the issues of selection and discrimination. Just for the record, I am not trying to hide or shy away from any issues.
My goal is to establish the capabilities and limitations of GDTAAA and the WSPRnet data.
We have already established rules such as:
1. Exclude stations with only a 4 character Maidenhead Grid code.
2. Exclude WSPR links between stations in the same 2 character Maidenhead Grid area (10° latitude x 20° longitude).
3. Determine ground speed and track of an aircraft on the basis of multiple position indicators over time.
4. Ensure that the ground speed and track pass a consistency check using the aircraft performance parameters and operational conditions.
5. Predict the new position after two minutes first and then see if there is a close match to a position or progress indicator.
6. If there is no close match to a position or progress indicator, then assume the ground speed and track are unchanged for the next two minutes.
@Richard,
“Just for the record, I am not trying to hide or shy away from any issues.”
Nor was there any intention to imply anything to the contrary.
Your six point rules, or steps, are clearly set out above.
On a slightly separate matter:
Once your outbound tracking of VH-XNF (RSCU550) has reached the point of leaving the site of the distressed vessel, it may be of interest to “search” for the Hercules arriving with more lifeboats (my presumption, I agree).
You may have been able to “find” the C130J on the way out using larger than two minute steps, and then revert to two-minute stepping when it is in the region.
What I am alluding to is that if you demonstrate that the two aircraft can be “seen” at the same time on the same plot then this may be worthwhile. (And, more to the point, perhaps you will NOT have to track VH-XNF, RSCU550, all the way back to base in two-minute steps, if any). Your call, of course.
@All,
Continuing the validation test with the AMSA Challenger SAR jet (registration VH-XNF and call sign RSCU550) en-route to the search area to help the Indonesian fishing vessel D2S2 (Sortie 4), here is the output from GDTAAA from 04:22 UTC to 05:00 UTC:
https://www.dropbox.com/s/ge6uqinjm3860ui/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200422%20UTC.png?dl=0
https://www.dropbox.com/s/486rxiomx5uctys/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200424%20UTC.png?dl=0
https://www.dropbox.com/s/5lvq7m6bk8kv8wi/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200426%20UTC.png?dl=0
https://www.dropbox.com/s/74k6b088swbc9ks/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200428%20UTC.png?dl=0
https://www.dropbox.com/s/b4ivpamjzjkfbpe/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200430%20UTC.png?dl=0
https://www.dropbox.com/s/l9g8qsfnfrpuozt/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200440%20UTC.png?dl=0
https://www.dropbox.com/s/m05owhf62f0ia9n/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200448%20UTC.png?dl=0
https://www.dropbox.com/s/5pzxpgkr3styvlv/GDTAAA%20AMSA%20VH-XNF%20Sortie4%2014MAY2021%200500%20UTC.png?dl=0
@Richard. You have a position indicator at 0500 UTC, the previous 04:18? Is the rate of both indicators about what you expect?
@David,
I expect on average around 20% of the 2 minute steps have a position indicator, 20% have no indicator and 60% have a progress indicator.
On average the flight path, ground speed and track get reconfirmed or updated every 10 minutes with a position indicator. I also check the ground speed and track using the progress indicators.
After 04:30 UTC I was only reporting every 10 minutes whilst in a transit mode to the search area at a stable cruise speed and cruise altitude.
I simply reported 04:48 UTC rather than 04:50 UTC because there were no indicators at 04:50 UTC or 04:52 UTC.
At 04:48 UTC the aircraft was approaching the storm area and there was a head wind of 33 knots at 32,000 feet. The air speed of the aircraft was 439 knots which aligns with the long range cruise speed in still air of 436 knots for a Bombardier Challenger 604 jet.
@George G,
In addition to the 6 rules I mentioned yesterday, I have come up with rule 7 as a result of reviewing the GDTAAA results so far together with @David:
7a. Straight and level flight is characterised by on average: No Indicator = 20%, Progress Indicator = 60%, Position Indicator = 20%, where all Position Indicators are a maximum of 2 intersecting WSPR anomalous links.
7b. Turning, climbing or descending flight is characterised by on average: No Indicator = 20%, Progress Indicator = 30%, Position Indicator = 50%, where some Position Indicators can be more than 2 intersecting WSPR anomalous links.
@George G,
Here is a draft of rule 8. This is a rule regarding selection and discrimination.
8. The predicted position two minutes later is calculated based on the current estimated ground speed and track. Where there are no anomalous WSPR links passing within a justifiable distance of the new predicted position, the new predicted position becomes the current valid position.
When one or more anomalous WSPR links pass within a justifiable distance of the predicted position the current valid position is updated. A justifiable distance is defined as a small distance of uncertainty within normal operating conditions as follows:
8.1. A small change in ground speed due to a change in the wind at the new position and over the last 2 minutes as determined by interpolation of GDAS weather data at the assumed altitude of the aircraft.
8.2. A turn is in progress, but within the aircraft turning circle at a normal operational bank angle.
8.3. A climb or descent is in progress with an accompanying change in ground speed, but within a normal operational rate of climb (ROC) or rate of descent (ROD).
@George G,
Here is rule 9, which is more for traceability, transparency and error checking.
9. The position indicators for a given leg of a flight path will not be updated retrospectively once more accurate ground speed and track data is available.
Minor course jinks will result because previous estimated positions within a leg are not being updated. These jinks act as a kind of error bar along a flight path leg between the leg start point and the leg end point, when compared to a great circle path from the start to the end of the leg. Errors will occur due to the inherent inaccuracies within the system and result in small deviations from the end-to-end great circle flight path.
After a turn, a new leg will start and there will be a similar settling process along the new leg.