Converting Weather Data with The Weather Tool

Introduction

All distributions of ECOTECT include a version of The Weather Tool, a program that allows you to analyse and convert weather data for use in many of the analysis routines.

Unfortunately, accurate and comprehensive weather data is often difficult to obtain. Even when purchased in electronic form from official meteorological offices it can be incomplete and in the strangest of formats. To assist with this, The Weather Tool recognises a wide range of known file formats and even lets you tailor your own to match almost any ASCII format imaginable. A list of the automatically recognised formats is as follows:

For weather data to be useful in the analysis routines in ECOTECT, you really need a full year worth of hourly weather data. This cannot be monthly averages or even daily ranges, but actual data recordings for every hour of the day. The data required includes:

It is also desirable but not essential to include:

The main source of localised weather data is always going to be the local government weather station or the closest airport. We have tried to provide a range of data on the Square One website, however it is impossible to collect data for all the major population centres let alone smaller towns and cities. In case you do need to convert your own data, this tutorial runs you through the process using the two different types of data format.

Importing Fixed Format Data

Fixed format weather data refers to files in which particular parts of the data occupy specific columns in each line. Such formats are common where the Fortran programming language has been used. In these files the columns refer to characters along each line and each field is always of a fixed length.

In this tutorial we are going to use some example fixed format ASCII data which The Weather Tool does not automatically recognise. The files you need should all be in Tutorial Files folder in the ECOTECT installation directory, however you can also obtain them as a zipped archive by clicking here.

  1. To begin, open the  Weather Data Fixed Format ReadMe.txt  file in the Tutorial Files folder in the ECOTECT installation directory. You can do this in Notepad or by simply double-clicking it in Windows Explorer.



    You will usually get either a text file like this one included with any custom weather data to explain what the values in the file are and their units.
  2. With this file still open, run The Weather Tool .
  3. Click the Open button.

    In the Open Data File dialog box, change the Files of Type to Fixed Format Weather Files, then navigate to the Tutorial Files directory located in your main ECOTECT Install and open the Weather Data Fixed Format.dat file. The following dialog box will be displayed.



    The reason for choosing Fixed Format Weather Files is because the format of the data is determined by a fixed set of columns. That is, each value (regardless of whether the data is complete or not) will always start in a fixed column.
  4. With the dialog box shown above open, you now need to go through each set of values and specify what they are and their units.

    This is where the "Weather Data Fixed Format ReadMe.txt" file comes in, as you will need to refer to it for what each set of values are.
  5. Starting with the Month of the Year, make sure it's checkbox is ticked then left click and drag across columns 2 and 3 in the table above.

    As the columns in The Weather Tool begin at 0 instead of 1, you have to consciously subtract 1 from the character indexes in the description.

    You should also notice that the entire 2nd and 3rd columns become highlighted when you release the mouse button. With these selected click the Assign button, to assign the 2nd and 3rd columns to the Month of the Year field.

    Continue doing this for each successive data component.
  6. You can see that the first month value in the file is '1'. Thus you need to left-click in the Units column and select the 'Start at 1' option.

  7. When you get to Dry Bulb Temperature, you will notice that the text file states the columns run from 9 to 13, but there is only numbers in 10 to 12.

    Quite often data will be slightly mismatched or inconsistent. If this is the case you will need to exercise a bit of judgement as to which is correct (this may involve several attempts at importing).

    In this instance choose columns 10 to 12 anyway.
  8. Once the columns have been assigned it is important to specify the correct units.

    In this instance the value for Dry Bulb Temperature has three characters/components and the "Readme.txt" file says their units are in °C.

    The Weather Tool uses SI unit, therefore the Celsius is ok, we just need to reduce the value by one decimal point. You can always tell what units The Weather Tool is going to convert to as they are written next to the name of each component, in this case (°C).
  9. To change the scale for Dry Bulb Temperature, left click the units cell next to Dry Bulb Temperature. Then choose the Custom... item from the list.

    In the Custom Units Conversion dialog box, type in a value of 0.1



    This specifies that the Dry Bulb Temperature component must be multiplied by a value of 0.1 to give in degrees, resulting in a temperature of 19.0°C rather than 190°C for example.
  10. Continue specifying columns for each of the data components based on the information in the "Weather Data Fixed Format ReadMe.txt" file.

    The dialog box should eventually look similar to the image below.



    You will notice in this set of data there is no rainfall component. When a particular component doesn't exist in the data then you just need to make sure that it's check box is not ticked.
  11. Once all the columns and units have been defined, click the Import File button.

    Once imported, select the Hourly Data button on the left of the left panels. This should display something similar to the following:



    You can now save this data as a WEA file and load it into ECOTECT for use in the thermal and solar analysis of your models. You can also use the data analysis and visualisation features of The Weather Tool to gain a real understanding of what is going on in that climate.

Importing Separated Value Data

    Separated Value Files contain data in which the individual components are separated on each line by a specific character - either a coma, space, semi-colon or tab. As such, the field length can vary without affecting the readability of the file. This requires a substantially different importing method that fixed format data.

    As an example, we are going to use data exported from the Meteonorm program available from METEOTEST in Switzerland. This is a commercial program that contains an extensive database of world climate data from which it can interpolate average data for any location.

Importing Meteonorm Data into The Weather Tool

  1. The first step is to set up Meteonorm to export hourly output in you own custom format.

    Unless you own Meteonorm, a description of exactly how to do this would be quite pointless. If you do own the software, then it's pretty obvious how to do this using the Output Format item in the Format menu and choosing User Defined. This will display the following dialog box.

    The main point is that you need to know what each data field is and the units it is in before you can import it. Whilst you can use some trial and error, it is usually possible to trace the original data source and obtain the actual data format. We are using Meteonorm in this example so you can clearly see where the original data format came from.





    If you are using Meteonorm, set up the output format and user defined units as shown above and then generate and save an hourly data. Some example output is included in the ECOTECT Tutorials folder as "Meteonorm.dat".
  2. Run The Weather Tool and then click the Open button.

    In the File Open dialog box, set the type of file to Separated Data File and select the  Meteonorm.dat  file in the ECOTECT Tutorials folder. This will display the following dialog box.



    You may need to check that they look the same in order to verify your output units.
  3. Start at the top of the list and drag Hour of the Day (1-24) from the left hand list into the Column Value list.

    You can also do this by selecting any value in the left list and choosing the Add>> button, but dragging allows you to see where in the list the item will be inserted. If you wish to overwrite a value instead of inserting, simply hold the Control key down as you drag.
  4. After adding the first value, you have to set its units. You can see from the file that the first hour is given a value if '1'. Thus, you have to left-click in the Units column and choose the 'Start at 1' option.

  5. Continue dragging and setting the units of each field until your dialog box looks like the following. Note that the Global Radiation value is ignored. If you do not put in this field, the diffuse will become global, wind speed will be come diffuse, etc...



    The other issue to note is the type of separator characters to use. By default, a range of characters are shown, meaning that fields can be separated by either a coma, a semi-colon, a vertical bar, a full colon, a tab character (shown as a solid block) and a space (invisible in the text field but there at the end. In most cases leaving these characters as is will be fine - in our case we selected a coma in the Meteonorm output so we are ok too.

    However, if your data contains colons in a time value for example, you probably don't want those values considered as two separate fields so you can simply delete the colon character from the separator Characters edit box in the dialog.
  6. Once you added and ordered the fields as above, you can save these settings for use later.

     To do this simply select the Save button at the bottom-right of the dialog, select your own user directory in the File Save dialog that appears and create a new file called "Weather Data Tutorial.ccf", or whatever name you feel is appropriate.
  7. Once saved, select the Import File button at the bottom of the dialog box to import the data.

    If you have selected the HOURLY DATA button on the left of the main application window, you should see something very similar to the following.



    You can now save this data as a WEA file and load it into ECOTECT for use in the thermal and solar analysis of your models. You can also use the data analysis and visualisation features of The Weather Tool to gain a real understanding of what is going on in that climate.