Windy missed radiosonde for days tho data posted to NOAA archive
-
For Kuala Lumpur International Airport, KLIA, (48650), Windy fails to report and carry in its 14-archive the data for many days in a week, but the data is available from NOAA's global radiosonde archive. Kuantan airport (48657) is likewise. See below. In the first screenshot, on the right is the text file I opened from the archive downloaded from NOAA. For each station, the year-to-date archive can be found at https://www.ncei.noaa.gov/data/integrated-global-radiosonde-archive/access/data-y2d/ , and KLIA data is in MYM00048650-data-beg2021.txt.zip. You can find KLIA has radiosonde data for twice a day, 7 days a week, and Kuantan is once a day, 7 days a week.
-
@Ondřej-Šutera
This missing radiosonde ascent data issue has much improved but this morning missing Radiosonde data happened again. See screenshot below. All the other radiosonde stations on the Malay Peninsula and Singapore reported ascent data 8h ago, but not KLIA, whose newest data is from the day before (Jul 31) rather than today Aug 1 0Z.
-
@hangInThere If I'm not mistaken, as for now, newest data for KLIA in NOAA archive are from 2023-07-30 12:00 UTC (source file):
Ascent for KLIA currently available on windy is 24h newer and came from NOAA GTS
-
@Filip_K Normally, all radiosonde stations' ascent data are available on Windy within 4 hours, whereas NOAA's archive only show them up a day later. So I do not think Windy sources radiosonde data from NOAA's archive. Please check your current source of radiosonde ascent data for the Malay Peninsula. Does Windy receive data directly from MetMY?
The NOAA archive is useful if Windy's 14-day radiosonde has missing data in which case the missing gap can be plugged with data from NOAA archive, which I did after posting the first message in this thread by adapting a tutorial Python script from NOAA.
-
@hangInThere said in Windy missed radiosonde for days tho data posted to NOAA archive:
Does Windy receive data directly from MetMY?
Negative, we don't have separate source for Malaysia. According to metadata, ascents come from NOAA (GTS and MADIS) feeds of aggregated ascents
-
@Filip_K Saw this buggy radiosonde this morning. Happened many times and reported many times to this forum but still happens. Could you take a look?
-
@Filip_K Windy Radiosonde is missing data from Banda Aceh (96011), Medan (96035), Pangkal Pinang (96237) and Ranai (96147) for days. But the ascent data for all these sites are found in said NOAA radiosonde archive everyday recently up to 1 August 2023. The meta data shows data source is NOAA MADIS, unlike the Malay Peninsula which show NOAA GTS.
I suggest
(1) switch the data source to NOAA GTS
(2) for past 14 day data except 1 to 2 days ago, pull from NOAA archive. I modified this tutorial https://unidata.github.io/siphon/latest/examples/upperair/IGRA2_Request.html to plot below skew T chart for Ranai 8PM 1-August.
-
@hangInThere we're aware of deformed skew-T diagrams and we're working on solution.
Regarding your suggestions:
(1) switch the data source to NOAA GTS
We're using both of NOAA sources and they are combined in a way that whichever ascent comes first is used. So switching to single source is not necessary.
(2) for past 14 day data except 1 to 2 days ago, pull from NOAA archive...
Thank you, I will look into fetching missing ascents from the archive. However, this probably won't be one of our top priorities right now so I ask for your patience
-
@Filip_K Radiosonde data are again missing today (2023-08-10 0Z) for KLIA (48650) and Penang (48601). Used to happen many days in a row back in July but has been really good in the past 2 weeks. What gives again?
-
@Filip_K Regarding (1), Windy still misses data for those stations (96035 Medan 96147 Ranai 96011 Banda Aceh 96237 Pangkal Pinang) on Sumatera and Ranai Island. OTOH, NOAA GTS realtime archive (https://ruc.noaa.gov/raobs/, info: https://ruc.noaa.gov/raobs/General_Information.html) provides today's ascent data, as below. If Windy is taking whichever comes in first btwn NOAA-MADIS and NOAA-GTS, perhaps what it receives from MADIS is blank. In fact I have seen this yesterday looking thru MADIS's RAOB directory and specifically looking for 48650 (KLIA) and it was all blank except for wind direction & speed. I didn't look for the Sumatera stations. I suggest you EITHER (A) drop MADIS altogether, OR (B) edit your source code to look out for blanks.
254 0 10 AUG 2023 1 99999 96011 5.52N 95.42E 21 2337 2 500 921 1060 50 99999 3 3 9999 63 kt 9 10090 21 238 221 70 3 4 10000 102 236 229 120 5 5 9780 295 246 211 99999 99999 5 9320 713 230 185 99999 99999 4 9250 785 224 180 245 18 5 9130 898 218 177 99999 99999 4 8500 1518 190 159 240 2 5 8100 1928 172 132 99999 99999 5 7940 2097 158 125 99999 99999 5 7390 2699 120 113 99999 99999 5 7170 2950 114 78 99999 99999 4 7000 3164 100 72 10 5 5 6170 4195 32 20 99999 99999 5 5770 4732 -9 -9 99999 99999 5 5630 4927 -9 -45 99999 99999 5 5270 5450 -37 -37 99999 99999 5 5060 5769 -51 -92 99999 99999 4 5000 5880 -57 -88 115 11 5 4790 6214 -77 -85 99999 99999 5 4670 6411 -79 -125 99999 99999 5 4510 6680 -93 -153 99999 99999 5 4310 7028 -119 -169 99999 99999 5 4070 7463 -143 -313 99999 99999 4 4000 7600 -153 -283 110 17 5 3890 7809 -169 -259 99999 99999 5 3740 8102 -183 -323 99999 99999 5 3130 9400 -287 -397 99999 99999 4 3000 9710 -309 -399 90 22 5 2770 10270 -347 -437 99999 99999 5 2690 10473 -365 -455 99999 99999 5 2560 10813 -397 -412 99999 99999 4 2500 10980 -411 -423 105 30 5 2240 11715 -471 -477 99999 99999 4 2000 12450 -533 -558 120 31 4 1500 14240 -683 -733 130 37 5 1430 14524 -709 -759 99999 99999 5 1170 15687 -785 -865 99999 99999 4 1000 16590 -734 -772 95 38 5 995 16619 -727 -765 99999 99999 5 854 17507 -754 -834 99999 99999 5 836 17631 -711 -781 99999 99999 5 749 18283 -689 -839 99999 99999 4 700 18690 -669 -829 190 11 5 650 19140 -639 -839 99999 99999 4 500 20740 -645 -925 150 15 5 458 21273 -657 -937 99999 99999 254 0 10 AUG 2023 1 99999 96035 3.57N 98.68E 25 2330 2 300 1000 492 45 99999 3 3 WIMM 99999 kt 9 10100 26 246 235 160 4 5 10030 85 244 234 99999 99999 4 10000 95 248 232 145 10 5 9980 112 256 230 99999 99999 5 9540 505 244 209 99999 99999 4 9250 779 226 194 115 3 4 8500 1518 182 161 75 1 5 7860 2178 134 131 99999 99999 4 7000 3157 94 64 205 4 5 6870 3312 88 56 99999 99999 5 5670 4868 0 -60 99999 99999 5 5520 5081 -15 -31 99999 99999 4 5000 5880 -57 -60 100 16 5 4350 6956 -113 -126 99999 99999 4 4000 7600 -153 -168 115 12 5 3380 8848 -235 -263 99999 99999 4 3000 9710 -299 -331 85 18 5 2670 10526 -367 -397 99999 99999 4 2500 10980 -403 -453 85 23 5 2450 11117 -411 -471 99999 99999 5 2180 11896 -481 -515 99999 99999 4 2000 12460 -531 -568 115 27 5 1800 13128 -591 -622 99999 99999 5 1650 13666 -635 -685 99999 99999 4 1500 14250 -681 -741 95 34 5 1420 14576 -704 -764 99999 99999 5 1270 15229 -751 -789 99999 99999 5 1100 16048 -811 -843 99999 99999 5 1070 16203 -821 -849 99999 99999 4 1000 16580 -779 -839 85 24 5 967 16774 -719 -809 99999 99999 5 890 17260 -734 -794 99999 99999 5 825 17702 -739 -809 99999 99999 5 770 18107 -705 -795 99999 99999 4 700 18680 -705 -795 210 12 5 689 18774 -707 -797 99999 99999 5 598 19627 -631 -751 99999 99999 4 500 20730 -675 -765 165 3 5 318 23477 -631 -771 99999 99999 4 300 23820 -647 -777 85 25 5 285 24133 -639 -799 99999 99999 254 0 10 AUG 2023 1 99999 96147 3.95N108.38E 2 2330 2 1500 1770 99999 37 99999 3 3 WION 63 kt 9 10130 2 290 279 250 2 4 10000 120 260 222 185 5 5 9800 296 254 219 99999 99999 4 9250 806 228 198 260 6 5 9150 900 222 192 99999 99999 4 8500 1539 180 157 285 9 5 7960 2094 148 137 99999 99999 5 7690 2383 130 126 99999 99999 4 7000 3179 96 55 360 4 5 6580 3688 70 -20 99999 99999 5 6180 4198 40 -20 99999 99999 5 6030 4396 26 -12 99999 99999 5 5660 4904 4 -76 99999 99999 5 5470 5176 -15 -45 99999 99999 4 5000 5900 -53 -103 100 12 5 4700 6382 -75 -155 99999 99999 5 4490 6735 -105 -142 99999 99999 5 4100 7429 -129 -279 99999 99999 4 4000 7620 -145 -325 115 14 5 3820 7966 -167 -377 99999 99999 5 3590 8428 -203 -293 99999 99999 5 3440 8742 -223 -403 99999 99999 5 3170 9335 -269 -419 99999 99999 4 3000 9730 -307 -417 105 6 4 2500 11000 -409 -479 85 16 4 2000 12480 -537 -576 65 29 5 1950 12642 -551 -580 99999 99999 5 1770 13252 -597 -647 99999 99999 5 1710 13467 -593 -663 99999 99999 4 1500 14270 -661 -731 60 24 5 1420 14599 -689 -739 99999 99999 5 1280 15211 -739 -839 99999 99999 5 1110 16026 -807 -887 99999 99999 254 0 10 AUG 2023 1 99999 96237 2.17S106.13E 33 2330 2 200 0 1090 76 7 3 3 WIKK 99999 kt 9 10100 33 244 217 180 3 5 10010 111 254 231 99999 99999 4 10000 119 254 231 140 10 5 9650 429 236 217 99999 99999 5 9420 637 220 199 99999 99999 5 9310 738 212 187 99999 99999 5 9290 756 212 183 99999 99999 4 9250 804 218 138 110 21 5 9230 823 226 106 99999 99999 5 9200 851 234 44 99999 99999 5 9180 870 234 44 99999 99999 5 9020 1022 216 136 99999 99999 5 8930 1108 210 150 99999 99999 5 8870 1166 208 138 99999 99999 5 8800 1234 218 28 99999 99999 5 8710 1323 222 -58 99999 99999 4 8500 1535 206 -84 110 9 5 8200 1841 172 52 99999 99999 5 8150 1893 164 104 99999 99999 5 7980 2071 148 115 99999 99999 5 7710 2360 134 64 99999 99999 4 7000 3173 96 26 25 8 5 6490 3793 56 6 99999 99999 5 6270 4073 50 -40 99999 99999 5 6080 4322 32 -38 99999 99999 5 6000 4429 24 -36 99999 99999 5 5890 4578 22 -158 99999 99999 5 5630 4940 14 -346 99999 99999 5 5440 5215 0 -140 99999 99999 5 5370 5318 -9 -119 99999 99999 5 5200 5573 -21 -261 99999 99999 4 5000 5890 -47 -167 120 10 5 4740 6305 -87 -167 99999 99999 5 4610 6519 -107 -167 99999 99999 5 4580 6569 -113 -163 99999 99999 5 4570 6586 -113 -173 99999 99999 5 4510 6687 -97 -297 99999 99999 5 4480 6738 -87 -447 99999 99999 5 4470 6755 -87 -467 99999 99999 4 4000 7620 -145 -445 95 20 5 3880 7850 -157 -467 99999 99999 5 3070 9562 -299 -519 99999 99999 4 3000 9730 -307 -527 125 27 5 2740 10367 -347 -577 99999 99999 4 2500 11000 -405 -575 90 18 4 2000 12480 -541 -591 75 22 5 1980 12544 -545 -594 99999 99999 5 1670 13608 -637 -697 99999 99999 5 1530 14141 -657 -757 99999 99999 4 1500 14260 -667 -777 90 49 5 1260 15290 -749 -829 99999 99999 5 1170 15716 -777 -837 99999 99999 5 1090 16119 -791 -861 99999 99999 4 1000 16620 -779 -869 85 30 5 965 16827 -709 -869 99999 99999 5 907 17195 -685 -855 99999 99999 5 837 17672 -714 -884 99999 99999 5 754 18286 -721 -921 99999 99999 4 700 18740 -689 -919 275 26 5 667 19030 -667 -917 99999 99999 5 638 19297 -683 -913 99999 99999 5 609 19577 -661 -941 99999 99999 4 500 20770 -677 99999 260 36 5 475 21077 -681 99999 99999 99999 5 353 22866 -655 -975 99999 99999 5 341 23077 -625 -955 99999 99999 4 300 23890 -611 -941 60 31 5 260 24781 -587 -927 99999 99999 5 224 25726 -535 -885 99999 99999 4 200 26450 -561 -911 85 53 5 199 26482 -563 -913 99999 99999 5 150 28296 -505 -865 99999 99999 -
@hangInThere great, thank you for thorough investigation. I'll look into filtering the data from MADIS when I get some other things out of my plate
-
@Filip_K Kuantan (48657) is missing radiosonde ascent data on Windy this morning 24 Aug 2024 0z, BUT NOAA ESRL (what I discussed last time) has it, which is weird b/c Windy supposedly gets from NOAA GTS, and usually if a station is missing data it happens to both, but today it happens to Windy only. Something gives?
-
@Filip_K This problem suddenly impacts most stations on Peninsular Malaysia, a sudden deep deterioration. There's no data from all but one from this morning 2023-09-15 0Z on Windy, which supposedly sources data from NOAA GTS, whereas I can download and display data no problem from NOAA realtime archive for ALL four stations (See below).
-
@Filip_K Radiosonde ascent data from Penang (48601) and Kuantan (48657) have been missing from Windy for 6 and 4 days, respectively. They are available from NOAA's realtime database. I suggest Windy adds this realtime database as an alternative source of data.
-
@Filip_K Radiosondes for Indonesian airports at Medan, Banda Aceh, Pontianak and Ranai are no longer accessible on Windy mobile. Are they permanently excluded? If so, why? The daily 0Z and 12Z data are available in real time from ESRL database I showed previously. Below is what I plotted from Bangka Island's Pangkal Pinang airport radiosonde for 0Z today:
-
@hangInThere
It looks like all Indonesian radiosonde ascents are currently missing from all our data feeds. We'll most likely add NOAA archive as source during an import rewrite, which is currently in progress -
@Filip_K
NOAA has two separate archives: IGRA under NCEI which stores ascent data at least 1 day old, and realtime RAOBS under ESRL which has up-to-date ascent data. Make sure you poke into the latter. -
@Filip_K There's been no update on Windy to Malaysia airports' radiosonde for 3 days.
I also checked ESRL (haven't done so for a year) and was disappointed they have cut off visits from the public. -
Hi @hangInThere, sorry for delay in response. There was an issue in our ingress pipeline. All should be caught up now
-
@Filip_K Yeah I saw this morning those airports have today's Radiosondes! Missed them much.
