In this case study we analyse the tragic flight of a Cessna 551 Citation II/SP registration OE-FGR, which crashed after fuel exhaustion into the Baltic Sea off the coast of Latvia on Sunday 4th September 2022.
The aircraft departed Jerez-La Parra Airport (XRY) in Spain at 12:56 UTC, en route to Cologne Airport (CGN) in Germany. The aircraft continued at FL360 through German and Swedish airspace. The aircraft was not reachable by air traffic control authorities for some time. French, German and Danish fighter jets were in turn dispatched to follow the aircraft. Reportedly the pilots could not see anyone in the cockpit of the aircraft.
This case study is similar to the tragic flight of MH370 in as much as the aircraft flew until fuel exhaustion and then crashed into the sea. The airspace around the aircraft was cleared of other air traffic over the Baltic Sea with the exception of the fighter jet assigned to follow and observe the Cessna.
The analysis in the report supports our previous belief that using WSPRnet data to detect and track MH370 together with the Boeing 777-200ER performance data and the Inmarsat satellite data provides a reliable method to determine the crash location.
The case study can be downloaded here
Updated 30th October 2022 to include on page 1: “The last 22 minutes of flight were chosen because the airspace in the vicinity of Cessna was cleared of other traffic. This was not the case for the flight before reaching the Baltic Sea.”
@All,
Geoffrey Thomas has published a new article on MH370 and WSPRnet technology:
https://www.airlineratings.com/news/mh370-new-research-paper-confirms-wsprnet-tracking-technology/
The authors said that they “have demonstrated how aircraft can be detected and tracked both in the cruise phase of a flight in straight and level flight as well as in the descent phase whilst descending and turning.
The authors also investigated alternative hypotheses and anomalies in relationship to WSPRnet links and said that while the WSPRnet data was noisy, with care it is possible to extract useful information.
They add: “The analysis in the report supports our previous belief that using WSPRnet data to detect and track MH370 together with the Boeing 777-200ER performance data and the Inmarsat satellite data provides a reliable method to determine the crash location.
@All,
Prof. Simon Maskell asked us to produce some statistics comparing the latest case study to previous findings on MH370.
Case Study Cessna OE-FGR.
183 Observations.
119 Unique Triples (Tx-Rx-Frequency).
109 Unique WSPR Links (Tx-Rx).
78 Unique Tx.
78 Unique Rx.
Mean Great Circle Distance Tx to Aircraft = 2,592 km (Min. 284 km, Max. 10,273 km).
Mean Great Circle Distance Tx to Rx = 1,633 km (Min. 175 km, Max. 15,567 km).
Case Study MH370.
187 Observations.
156 Unique WSPR Links (Tx-Rx).
92 Unique Tx.
79 Unique Rx.
Mean Great Circle Distance Tx to Aircraft = 13,676 km (Min. 1102 km, Max. 19,868 km).
Mean Great Circle Distance Tx to Rx = 4,430 km (Min. 510 km, Max. 16,527 km)*.
* = only WSPRnet Links with Tx to Rx Great Circle Distance ≥ 500 km were considered, hence the Min. is 510 km.
@All,
There were 23,871 WSPRnet links in the 24 hour period with a propagation ≥ 15,000 km on 4th September 2022:
https://www.dropbox.com/s/yugipq1ccmegj66/Count%20WSPRnet%20Links%20%E2%89%A5%2015%2C000%20km.png?dl=0
There were 5,596 WSPRnet links from transmitter call sign VK3MO in a 24 hour period with a propagation ≥ 15,000 km on 4th September 2022:
https://www.dropbox.com/s/9k6gb3bfg6d7r7n/Count%20Tx%20%E2%89%A5%2015%2C000%20km.png?dl=0
The greatest propagation distance recorded in the 24 hour period on 4th September 2022 was 19,769 km between ZL2TLF and EA8CHC.
https://www.dropbox.com/s/0afepjqkth9vujt/Max%20Distance%20WSPRnet%20Links.png?dl=0
@All,
Interesting Propagation Case from the current Case Study.
An interesting propagation case is from transmitter (Tx) call sign OZ7IT at locator JO65df on the Baltic Sea coast of Denmark, which was 481.3 km from the aircraft position at 17:26:22 UTC on an initial bearing azimuth of 61.4938°T.
At 17:38:00 UTC receiver (Rx) call sign VK4CT at locator QG62jv at Rocksberg, Queensland, Australia receives a WSPRnet protocol from OZ7IT with a SNR of -18 dB.
The great circle path from OZ7IT to VK4CT is on an initial bearing azimuth of 61.4938°T, so the aircraft crossed the great circle path between Tx and Rx.
The received SNR of -18 dB is an anomaly at 0.95 SD when compared to the 4 WSPRnet links received on the 4th September 2022.
I discount this link over the timeframe ±3 hours because the count was only 3.
I looked at ±1 day ±3 hours and the count was 25, but -18 dB was no longer regarded as an anomaly at 0.58 SD.
I then noticed at 17:28:00 UTC VK4CT receives a WSPRnet protocol from OZ2BR at locator JO65di with a SNR of -14 dB, which is an anomaly of 1.21 SD over a ±3 hours timeframe.
I discounted this link it passes 2.5 nmi from the aircraft and is outside my 2.0 nmi circle of uncertainty.
@Richard
Excellent work by you, Dr. Hannes Coetzee, and Prof. Maskell, and courageous reporting by Geoffrey Thomas. The WSPR crash site fits within the UWA drift analysis recommended search area. Brace for incoming from the detractors, but we will not duck and cover, and never run!
Search On !
@All,
Two comments on the recent case study by Prof. Simon Maskell:
“The analysis involves a Cessna over Europe, rather than a 777 over the Indian Ocean, and, while promising, as a result of the relatively small dataset considered, is not definitive.
However, it is also clear to me that the analysis implies it is probable WSPR can detect the Cessna better than can be explained by chance.
It is therefore likely that an algorithm could exist that can track the aircraft’s location by processing the detections derived from the raw WSPR data.”
… and …
“I think what you have said in the report is logical and statistically correct. My advice is that you should take these results to Ocean Infinity.”
We sent our case study to Ocean Infinity yesterday for their review.
@All,
Victor Iannello, Mick Gilbert and Steve Kent all claim to have read our latest case study but somehow miss one of the most important points.
“The plot of the True Positive Rate against the False Positive Rate for the Receiver Operating Characteristic (ROC) curve during the cruise phase of the flight shows an improved level of aircraft detection. The ROC curve is based on the 70 observations during the cruise and the area under the curve increases from 58.66% to 60.43%.” – Discussion of the Results – Section 7.2 – (please see Figure 52).
Victor Iannello concludes:
“So by my count, with a threshold of 0.75, after including the discarded data, the True Positive rate remains at 52% and the False Positive rate increases from 37% to 44%, versus a rate of 45% of detections due to chance (assuming that SNRs are normally distributed).”
“The utility of the GDTAAA plane detector just went from weak to almost imperceptible, just as physics would predict. The prospect of using this detector to reconstruct the path of MH370 is once again proven to be junk science.”
Mick Gilbert concludes:
“Victor, doubtless Professor Maskell is familiar with the traditional academic point system for classifying the utility of a methodology based on its area under the ROC curve results;
.90-1.0 = excellent (A)
.80-.90 = good (B)
.70-.80 = fair (C)
.60-.70 = poor (D)
.50-.60 = fail (F)
With an overall AUROC of 58.66% the authors get an F.”
“We might well ask whether anyone in any position of authority on either side of the negotiations to fund a renewed search would treat a methodology with essentially no predictive capability seriously. Who could possible say that a methodology with such manifestly poor results clears the “credible” part of the 2017 “credible new information” test?”
Steve Kent concludes:
“Godfrey takes the unjustified step of discarding duplicates, retaining only the links that cross the true aircraft position. Since the majority of the discarded links are positive detections (21 out of 28), this means that the retained links are mainly true positives while the discarded links would have been false positives had they been kept. This process introduces a bias such that the WSPR hypothesis will always be favoured, even if the null hypothesis were true. A better designed test would have avoided this issue altogether”.
“This analysis ignores the fact that rays do not follow precise great circles (as assumed by Godfrey) due to the spheroidal shape of the earth, introducing cross-track offsets of order 10-15 nm and thus making the entire exercise total nonsense.”
In response we would like to point out the following:
1. Independent Review.
Prof. Simon Maskell is on the record as stating: “the report is logical and statistically correct.”
2. Discarding Duplicates.
The results in the cruise even with the duplicates is the same at 58%.
https://www.dropbox.com/s/wdql880ybvccuz9/ROC%2073%20Cruise%20Observations%20with%20optimal%20operating%20point%20and%20confidence%20intervals.png?dl=0
The duplicates arise mostly from the spiral dive, where the aircraft in the last 4 minutes of flight is all within a circle with a radius of 2 nmi, and also in part from the alignment of the aircraft track with the WSPRnet link azimuth.
3. Methodology Classification.
Gilbert invents a new 5 point Likert rating scale. He probably meant “Excellent, Very Good, Good, Fair, Poor”. There is no “F” for “Fail” in a Likert scale. We also note the next letter in the alphabet after “D” is in fact “E” and not “F”. Neither Prof. Maskell, nor we, have heard of this Gilbert rating scale.
In fact a result of 60.43% is “Fair” and not “Fail”.
4. Poorly Designed Test.
The tragic flight of a Cessna 551 Citation II/SP registration OE-FGR, was not a poorly designed test.
In fact the aircraft that crashed after fuel exhaustion into the Baltic Sea off the coast of Latvia on Sunday 4th September 2022 was not a test at all.
Like MH370 it was an incident that should not have happened and any investigation into the incident has the prime goal to prevent such an incident ever happening again.
5. Cross-Track Errors.
Steve Kent states: “I computed the rms cross-track error for each of the 183 links in the data table. The median rms is about 2.1 nm, which is the maximum cross-track error allowed by Godfrey”
Steve Kent omits to state that the average number of SNR anomalous WSPRnet links at each point in the cruise of the aircraft is 4.3 and not a single link as he implies. When three or more anomalous links intersect at the aircraft position, as is the case at each point in the time frame analysed, then the cross-track error is minimal.
It seems that there was some reference to the younger pilot reacting to something aboard this MH370 plane after take-off . That he left the cockpit to use his cell phone , something like that . Has anyone guessed at how the older pilot disabled the younger pilot and how much into the flight ? How would he have been disabled , possibly shot with a gun ???
@Barbara Roth,
Welcome to the blog!
There is no evidence that the co-pilot reacted to something on board MH370 after take off that I am aware of.
There is evidence that his mobile phone was switched on and detected by a mobile phone provider at a tower on the island of Penang.
It is possible if the co-pilot leaves the cockpit to lock the cockpit door and prevent reentry.
It is difficult to take a gun onboard an aircraft due to the security screening.
Although my qualifications may not compare to those who have contributed to the search for MH370, I would like to express my observations regarding the similarities between the 1999 South Dakota Learjet crash and the MH370 case.
It is plausible that the aircraft may have flown unmanned until fuel exhaustion, which would provide answers to several questions, such as why families were able to connect to their loved ones’ phones and why the wreckage remained in one piece.
In my opinion, the pilot may have become aware of a fault in the two minutes following the last contact, significant enough to make a U-turn back to Kuala Lumpur International Airport.
If the fault pertained to the communication systems, it could explain the movements on the military radar, as the pilots were likely attempting to complete checklists and re-establish communication with air traffic control. Meanwhile, cabin decompression may have led to a fate similar to that of the 1999 Learjet crash.
Overall, I believe that the efforts of those involved in the search, particularly those who have persevered despite the passage of time, are commendable.
@Lewis Joyce,
Welcome to the blog!
Different mobile phone operators handle a call to a mobile phone that is not connected to a base station anywhere in the global network of possible providers differently. Sometimes you will get a message that your call cannot be connected, sometimes they will just provide a ringing tone to make you think that your call could be connected.
The wreckage did not remain in one piece. We have recovered 39 pieces with an average weight of 4.6 kg. The zero fuel weight of the aircraft was 174,369 kg. If the aircraft fragmented at the same rate as the debris items recovered so far, then there would be 37,906 pieces.
if the pilot became aware of a fault serious enough to cause a turn back, then he would perform an emergency landing at the nearest airport. There are 3 VHF radios, 2 HF radios and a satphone in the cockpit. It is unlikely that all communications systems had failed. In an emergency the active transponder would be set to 7777. Instead both transponders were switched to standby and the ACARS system which would give position reports every 30 minutes was switched off.
The pilot was still active long after the diversion had taken place. If the landing gear was lowered towards the end of the flight, then this would require an active pilot. The aircraft made changes of speed and altitude. Turns can be programmed into the Flight Management Computer but changes of speed and altitude cannot be pre-programmed and require an active pilot. The Learjet in 1999 flew in straight and level flight at a constant airspeed until fuel exhaustion as there was no active pilot.
What puzzles me: There must have been other radio signals of passenger’s mobile phones been caught somewhere. I’ve never seen such data.
The WSPR data is interesting, I do not understand the statistics behind the geolocation to be honest.
Vy 73, DK1CAB
@Dr. Hinner,
Welcome to the blog!
We have discussed the issue that only the mobile phones from the crew were analysed and only over Malaysia. There were many more mobile phones from passengers that could have been analysed and not only over Malaysia. I am currently looking into trying to get Indonesian mobile phone data from base stations on off shore rigs.
Here are some of the most relevant comments on the subject of mobile phone detection:
https://www.mh370search.com/2023/12/18/wspr-as-radar/comment-page-1/#comment-2356
https://www.mh370search.com/2023/12/18/wspr-as-radar/comment-page-1/#comment-2372
https://www.mh370search.com/2023/02/26/the-ongoing-search-for-mh370/comment-page-1/#comment-2164
https://www.mh370search.com/2023/02/26/the-ongoing-search-for-mh370/comment-page-1/#comment-2162
https://www.mh370search.com/2022/09/08/mh370-detection-and-tracking/comment-page-1/#comment-2150
https://www.mh370search.com/2022/03/14/mh370-wspr-technical-report/comment-page-2/#comment-2036
https://www.mh370search.com/2022/09/08/mh370-detection-and-tracking/comment-page-1/#comment-1992
Our hypothesis with the WSPR data comprises both a null hypothesis and an alternate hypothesis. In other words, we are trying to show in the alternate hypothesis that WSPRnet data can be used to detect and track an aircraft when the aircraft position is at the intersection of the great circle paths between WSPR transmitters and receivers. At the same time, we are trying to show in the null hypothesis that WSPRnet data will not detect an aircraft when the aircraft position is not at the intersection of the great circle paths between WSPR transmitters and receivers.
The null hypothesis is: WSPRnet links show statistically significant anomalies when an aircraft is not on the great circle path between the transmitter and receiver.
The alternative hypothesis is: WSPRnet links show statistically significant anomalies when an aircraft is on the great circle path between the transmitter and receiver.
The objective of the papers and case studies we have published on this website is to demonstrate a statistically significant number of examples of the alternative hypothesis and show that the alternative hypothesis is generally true, whilst at the same time demonstrate a statistically significant number of examples of the null hypothesis and show that the null hypothesis is generally false.
I think it is hard to argue that a Receiver Operating Characteristic (ROC) curve that is distant from the y=x line is measuring pure noise. We presented 7 such ROC curves in our case study titled “Flight QTR901 GDTAAA WSPRnet Analysis” published 8th June 2023.
The plot of the True Positive Rate against the False Positive Rate is a standard test in statistics and is called the ROC curve. It is a test of sensitivity and specificity. Sensitivity (true positive rate) refers to the probability of a positive test, conditioned on truly being positive. Specificity (true negative rate) refers to the probability of a negative test, conditioned on truly being negative.
We are about to publish a set of ROC curves with a larger number of observations taken from our latest paper titled “How does WSPR detect Aircraft over long Distances?” and published in the post on WSPR Aircraft Tracking.
My co-author Prof. Simon Maskell is currently working on a large scale study using around 1,000 Boeing 777 airborne during a 24 hour period, where he has the ADS-B data and WSPRnet data for all aircraft globally (not just the Boeing 777’s) at each of the 720 two minute WSPR time points.
He plans to use the results in a second study on the MH370 flight path using the processing chain for MH370 defined by the DSTG particle filter in their paper titled “Bayesian Methods in the Search for MH370” but revised to include the WSPR data.
@Richard Godfrey
Thank you for your reply. Would it be possible that I can try to follow your path of calculus with your raw data? Is it in R format? I can also work with SPSS or any other statistics file format basically.
Thank you and all the best,
Vy 73, DK1CAB, Dr. Hinner, München, Bayern
@Dr. Hinner,
I have an automated processing chain using Matlab. I call the software Global Detection and Tracking of Any Aircraft Anywhere (GDTAAA). The software is based on the WSPRnet data, which is available since 11th March 2008, so within the last 16 years it is Any Aircraft Anywhere Anytime.
The GDTAAA software automates the following process chain:
1. Generate the list of WSPRnet links for any given two minute WSPR timeframe from the WSPRnet database (SQL call to wspr.live).
2. Enrich each transmitter and receiver call sign with the actual antenna location at that date (sourced from ham websites and registries).
3. Check that the antenna location matches the Maidenhead grid code in the WSPRnet database.
4. Produce a list of any call signs missing from the antenna database, allowing for a database repair and rerun.
5. Generate the great circle path for each WSPRnet link (assuming isotropic antennas, a spherical Earth, no ionospheric or Earth surface topographic tilts).
6. Set the limits for Signal-to-Noise Ratio (SNR) anomalies and frequency anomalies (currently both set to 0.75 of a standard deviation from the mean).
7. Determine which WSPRnet links are anomalous over a defined timeframe from the WSPRnet database mean and standard deviation (currently set to ± 3 hours, using a Matlab web read to https://db1.wspr.live).
8. Determine which anomalous WSPRnet links intersect within a defined target area.
9. Track intersections of multiple anomalous WSPRnet links over time to see if the path follows a flight route or not.
10. Track intersections to see if the movement along a path is consistent with an aircraft’s ground speed.
11. From the ground speed determine the type of aircraft (helicopter, propellor aircraft, jet aircraft, combat aircraft).
12. Track intersections to see if the direction along a path is consistent with a destination airport, air base, airstrip, helipad or navigation waypoint on a flight route.
13. Track aircraft from any position and time without the need to know the last known position.
14. Track aircraft from any position to a destination at an airport, air base, airstrip or helipad.
15. Determine the position, ground speed and track at each detection along the flight path.
I am happy to share with you the Matlab scripts I use for personal academic research. The WSPRnet data is freely available for registered users. The antenna database is public domain information but requires confidential handling. The system is self documenting. An introduction session to using the software via Zoom takes around 1 hour from experience, for someone familiar with Matlab together with the Database, Mapping, Statistics and Machine Learning Toolboxes.
In addition I use PropLab Pro V3.2 for ionospheric ray tracing, global mapping of multi hop propagation, maximum usable frequencies, elevation angles and grey line checking.
@Richard Godfrey
About this claim:
”WSPRnet links show statistically significant anomalies when an aircraft is on the great circle path between the transmitter and receiver”
What is your manner of assessing statistical significance? Earlier, when I tried a simple t-test on the SNR values of both the QTR901 and the OE-FGR cases, the results for both were slightly above 0.05, which is the commonly used threshold for statistical significance (I can’t say this for certain since one value was missing from the tables and both cases were around 0.05-0.07).
Also, do you think the SNR anomaly values should follow normal distribution if they were completely random? Instead of using the actual control group values, I tried to explore what the ROC curve would look like when the SNR values were compared against a normally distributed group of 10,000 controls (to decrease the chance of the small control group being skewed). The AUC result I got in this way was about 0.535 for OE-FGR instead of the original result of 0.59.
I still think the results acquired in these case studies are too abnormal to be caused by random chance alone, so I look forward to the study by Maskell that will compare a large quantity of SNR values.
@Puuhöylä,
Welcome to the blog!
You ask: “What is your manner of assessing statistical significance?”
The null hypothesis is: WSPRnet links show statistically significant anomalies when an aircraft is not on the great circle path between the transmitter and receiver.
The alternative hypothesis is: WSPRnet links show statistically significant anomalies when an aircraft is on the great circle path between the transmitter and receiver.
The goal is to demonstrate a statistically significant number of examples of the alternative hypothesis and show that the alternative hypothesis is generally true, whilst at the same time demonstrate a statistically significant number of examples of the null hypothesis and show that the null hypothesis is generally false.
We are determining WSPR SNR anomalies by looking at all WSPRnet links for a particular Tx-Rx-frequency triple in the time frame ±3 hours. It is a reasonable assumption that the receiver software version will not change in this 6 hour timeframe. However over a 24 hour period there will likely be significant changes in the SNR and received Frequency, therefore we calculate the mean and standard deviation (SD) over a 6 hour period, ±3 hours either side of the test time point.
Each WSPR transmission contains the call sign of the transmitter, the location of the transmitter using a Maidenhead Grid code and the power level of the transmission. The receiver station augments this data with the call sign of the receiver, the location of the receiver using a Maidenhead Grid code, the received frequency, the frequency drift and the Signal to Noise Ratio (SNR) of the received signal. The Maidenhead Grid code is a 4 character code in the first transmission and a 6 character code in the second transmission. A 4 character code has a precision better than ±120 km and a 6 character code is better than ±5.2 km. Neither has the precision required to determine the great circle path between transmitter and receiver for the purpose of detecting and tracking aircraft. We therefore decided to build a database with transmitter and receiver antenna locations, where the precision of the latitude and longitude is given to 6 decimal places (better than 1 m).
Some WSPRnet stations are mobile in vehicles, ships or balloons. The SNR and Frequency anomalies over a 6 hour period between mobile transmitters or mobile receivers are discounted. These mobile stations can be detected from their call signs or the fact that the Maidenhead Grid locator changes during a six hour period.
We then check the count of each anomaly link over a ±3 hour period during the flight analysis timeframe, centred on the test datum. We discard candidate SNR or Frequency anomalies based on a small count or where the measured SNR or received Frequency is not more than 0.75 standard deviation from the mean during this analysis timeframe. We also discard drift anomalies where there is a continual drift during this analysis timeframe.
To assess whether a test datum is an anomalous link at a specific time on a specific day, we calculate the mean and standard deviation using data from a specific transmitterID-receiverID- frequency triple for a ±3 hour analysis window (centred on the test datum). If the count is below five we extend up to ±3 days, but for the same ±3 hours (centred on the test datum).
Signal level and signal frequency disturbances can result, when an aircraft flight path intersects with the propagation path of a WSPR link. This is evident from anomalies in either the SNR or the frequency drift over a short time period.
We use Matlab software for mapping the aircraft track and the anomalous WSPRnet links relevant to any particular aircraft detection and tracking. We use an online Vincenty calculator to determine the initial and final bearings of propagation paths between transmitter, aircraft and receiver. We use Proplab-Pro V3.2 software for HF Radio Propagation Ray Tracing produced by Solar Terrestrial Dispatch, which is a useful tool for tracing multi-hop ionospheric WSPR propagations. The great circle path of a radio wave is usually calculated using a spherical model of the Earth with a mean radius of 6,378,137.0 m. The actual height of the transmitting and receiving antennas is ignored in this calculation, as this usually only makes a small difference, except for relay stations at high altitude in the mountains.
We use the http://wspr.rocks/livequeries/ front end which exposes the WSPRnet database for SQL based queries. The WSPRnet database for any particular two minute timeframe is downloaded and exported to a spreadsheet.
We then augment the spreadsheet by adding columns of transmitter (Tx) latitudes and longitudes and columns with the receiver (Rx) latitudes and longitudes. These tables are derived from the master antenna locations database. We then match the tx_sign and rx_sign to derive the index tx_id and rx_id in each table.
Some call signs have more than one location, for example AF7XZ. We then simply manually changed call sign AF7XZ at location DM42 to AF7XZ-1 and AF7XZ at location FN00rm to AF7XZ-2. Any call signs which are not matched are then most likely mobile WSPR stations e.g. call sign SA6BSS/FLY, which is balloon or DP0POL, which is a ship. These rows can be deleted. We then use these indices to set the confirmed flags tx_c, rx_c and the tx_rx_c. We use these flags in the Matlab based processing of the SNR WSPRnet links to initially mark a link in red when both the Tx and Rx location are confirmed, or orange if only one is confirmed or grey if neither are confirmed.
If the antenna location is unconfirmed, then it means the 6 character Maidenhead grid locator is being used rather than the actual antenna position in the initial WSPR link processing. However, any unconfirmed link close to the area of interest identified during the processing will then be upgraded to a confirmed link and the final published results are only based on links with a confirmed antenna location. If the link is of interest, but the antenna location is unconfirmed, we then add the missing location data to the master database and rerun the process. This will move the great circle path of WSPR links in the Matlab mapping process slightly, if originally marked in orange or more significantly, if originally marked in grey.
We then use the Tx and Rx indices to append the tx_lat, tx_lon, rx_lat and rx_lon. Finally we generate a Matlab function call for each link which we paste into the Matlab script to generate the maps. Having filtered the links of interest and examined the SNR and drift anomalies over ±3 hour timeframe, we then reset the colours of the relevant WSPR links in the Matlab mapping script to red for SNR > 1.0 Standard Deviation (SD), orange for SNR > 0.75, cyan for Frequency > 1.0 SD, brown for Frequency > 0.75 SD, green for non-zero drift, blue for dual SNR and Frequency anomalies and grey for no anomaly.
Our Matlab process for call sign augmentation automatically adds confirmed latitude and longitude for each Tx and Rx. It also produces a list of any call signs missing from the antenna database.
Our Matlab process for WSPRnet links that pass within 100 km of an estimated aircraft position automatically produces a list of function calls for the Matlab mapping process.
The work being done by Prof. Simon Maskell, with a much larger dataset, will no doubt shed more light on ways to improve this process.
@Richard Godfrey
You’re going to have a 0.50+ result 50% of the time even with random chance alone, so you wouldn’t call a 0.51 result statistically significant with these sample sizes, right? My question was: what is the basis of making that claim with 0.57-0.59? As I said, I think the results are close to 0.05 in OE-FGR and QTR901 but not below.
@Puuhöylä,
We do not have a 0.51 result as you claim.
The two studies you quote regarding OE-FGR and QTR-901 show Receiver Operating Characteristics (ROC) curves with between 143 and 244 observations and an Area Under the Curve (AUC) of between 59% and 68%.
As we have pointed out in these two studies and elsewhere, “The points are all not that far from the y=x line. WSPR is therefore a noisy sensor. The 95% confidence interval error bars extend below the y=x line. The implication is that we need to process the anomalies carefully to extract useful information and we have developed a technique that aims to perform such careful extraction of useful information.”
I fully agree that 244 observations is too few. That is why I have also previously stated in a comment on this website in answer to @Karlo Timmerman on 9th May 2024, that we are building a ROC with around 500,000 observations:
https://www.mh370search.com/2024/05/05/new-search/comment-page-1/#comment-2719
“Our hope is that we can generate a ROC curve with, say, 1000 flights of data from Boeing 777s. If the results are as we anticipate based on the initial ROC curves, we feel it will be difficult to argue that WSPR does not provide compelling new evidence.
Simon has collected all the ADS-B data globally during a 24 hour period. There are on average 21,538 aircraft in the air at any one time. Simon has all the WSPR data globally for the same time period and has developed a processing chain to produce a ROC curve based on around 1,000 flights of a Boeing 777. Long haul aircraft like the Boeing 777 make on average 2 flights per day with an average flying time of 8.58 hours each. This gives around 500,000 rows of data for the ROC curve.
Simon should soon be able to provide a ROC curve that is based on the large amount of data that they have collated. Simon is very conscious that the conclusions from that work are likely to be contentious. They have overtly avoided being funded to do the work thus far. That should make it easier than it would be otherwise to argue that they are not being biased by any commercial considerations. Simon also plans for a peer review by 4 other academic institutions, who have expertise in Bayesian methods and are familiar with the DSTG work, their particle filter and ROC curves.
Assuming that the ROC curve indicates that WSPR has some utility, Simon anticipates a period of refining their algorithm, for example to hone their processing chain and/or analysis to understand how turns, climbs, descents, geography and solar weather impinge on the ROC curve.
Simon also plans to generate a revised search area based on using the statistics from the ROC curve to enable them to augment the analysis that the DSTG did previously, but this time to include WSPR. Simon acted as an adviser to the DSTG book on “Bayesian Methods in the Search for MH370”, which defines the particle filter they used to determine the ATSB search area. Simon plans to extend that particle filter to include the WSPR ROC curve from his research.”