IMPLEMENTATION OF A BATHYMETRY DATABASE FOR GEOSCIENCE AUSTRALIA FELLINGER Christian & KRUIMEL Daniel, CARIS Asia Pacific, Suite 1 Innovation House, Mawson Lakes Boulevard, Mawson Lakes, SA, Australia SPINOCCIA Michele, Geoscience Australia, Cnr Jerrabomberra Ave and Hindmarsh Drive, Symonston, ACT, Australia ABSTRACT Geoscience Australia (GA) is a world leader in providing first class geoscientific information and knowledge. This contributes to enhanced economic, social and environmental benefits to the community by providing input for decisions that impact upon resource use, management of the environment, and the safety and well-being of Australians. The Energy Division of GA is responsible for the management of bathymetry data from Australian Territorial waters and from a multitude of sources. There are a number of challenges in managing and interacting with data collected through different methods, with varying file sizes, accuracy and resolutions. To address these challenges, a new data management capability is required. GA commenced the task of implementing a relational database for managing its extensive bathymetry data. This task included dedicated on-site support to streamline workflows and the establishment of sound standard operating procedures in order to maximise the benefits of the new system. GA, with the support of CARIS, established a functional relational database. The Energy Division is now able to maintain their datasets in a secure, central database repository. The division has improved data discoverability with the availability of an extensive metadata catalogue. Data analysis capabilities have also been enhanced, including the ability to compare overlapping data collected through different platforms at given points in time. There is also an increased capability to efficiently provide data to other sections within GA. The new bathymetry database solution has provided GA's Energy Division with improved capabilities to manage and interact with their bathymetry data and share the data with other users at GA. GA & CARIS will continue to work closely together to ensure their mutual solutions meet the growing demands for the future management of bathymetry data. 1 INTRODUCTION Geoscience Australia is custodian of the geographic and geological data and knowledge of the nation, along with co-custodian of the largest collection of single and multibeam bathymetry data in the Australian territorial waters. As an Australian government agency within the Department of Resources, Energy and Tourism, GA is responsible for geoscientific research.
2 GEOSCIENCE AUSTRALIA OVERVIEW GA's role is to provide geoscientific information and knowledge which enables government and the community to make informed decisions about: the exploitation of resources the management of the environment the safety of critical infrastructure; and the resultant wellbeing of all Australians GA's activities can be divided into three sectors; onshore, offshore and spatial information. Onshore activities focus on enhancing mineral exploration and environmental land-use planning. This is achieved through the production of geoscience maps, databases and information systems, as well as conducting research into regional geological and mineral systems. More widely, onshore activities also contribute to safer communities and critical infrastructure along with the maintenance of fundamental gravity, geomagnetic and seismic networks Offshore activities focus on providing pre-competitive data and information to assist in identifying new prospective basins for petroleum exploration and the geological storage of carbon dioxide, within Australia's offshore jurisdiction. Activities also include mapping and documentation of Australia's maritime boundaries, studies of the marine environment using seabed mapping techniques and determining estuarine water quality and health. These studies assist in natural resource management. Spatial information activities focus on providing key spatial information of Australia with an emphasis on response to rapid and slow onset hazards, the detection of change, emergency management requirements, natural risk assessment and marine zone management. Furthermore, activities also include the coordination and implementation of the Australian Government's policy on spatial data access and pricing. Within GA s Energy Division, the Geophysical Analysis and Data Access project is responsible for the acquisition, processing and management of bathymetric data. 2.1 SURVEYS AND EQUIPMENT With over 325 surveys in their holdings, GA/GADA is co-custodian of the largest collection of single and multibeam bathymetric data in Australian territorial waters. These surveys have been acquired by a total of 43 vessels using a variety of bathymetric systems. The surveys can be roughly divided into three categories: 12 khz: For use in deep water (100m - 11km), EM120 (Vessels L'Atalante, Sonne, Roger Revelle) 30 khz: Intermediate depth systems (10m - 5km), EM300 (Vessels Southern Surveyor, Tangaroa) (see Figure2-1) 300 khz: Shallow water system (0.5m - 200m), EM3002D (transportable system used on vessels such as RV Challenger, Kimbla and the Australian Institute of Marine Science Vessel Solander) (see Figure2-2, Figure2-3) Figure 2-4 and Figure 2-5 show the distribution of surveys between the various platforms. As can be seen, the RV Southern Surveyor accounts for nearly one third of the total surveys. In total there have been 325 multibeam swath surveys, 315 of which have been converted into a single platform. The remaining surveys are from different sources and are of varying quality and have yet to be processed. In terms of file size, the surveys equate to 5.11 terabytes of the CARIS HDCS data structure, which includes a copy of the raw data. There are 1,422,378 line kilometres of data, containing 42.89 billion beams, and over 2 billion edited beams. The data covers an area
of 10,599,967 square kilometres in and around Australian waters. Fifteen percent of the Australian Exclusive Economic Zone is covered. An overview of these surveys are shown in Figure2-6). Figure 2-1: EM300 System Figure 2-2: EM3002D System Figure 2-3: The Southern Surveyor
Figure 2-4: Pie Graph of Surveys per vessel Figure 2-5: Tabular representation of Surveys per vessel
Figure 2-6: Geoscience Australia's Data Holdings over 2009 (250m bathymetric grid) 3 DATA MANAGEMENT With such a large amount of data from a variety of different sources, the challenge for GA was to convert all of these surveys into a single platform and be able to efficiently manage the data. Historically, the data has been archived on hard drives. A custom built HTML web browser was developed to provide a graphical interface to locate surveys. The user could enter a coordinate to discover what survey(s) existed at that location, with a reference to where the data was stored. GA wanted to improve their data management beyond this capability through the implementation of a database solution. 3.1 IMPLEMENTING THE DATABASE GA chose the CARIS Bathy DataBASE solution to assist them in their challenge of better managing such vast quantities of bathymetric data. The solution is comprised of both a client and server component. At the back end, the server application is running on a high specification PC, utilizing PostgreSQL as the relational database management system. The client application (BASE Manager) is used to compile and validate bathymetric information (both internal and external) so that it can be loaded into the database with a fully customizable and extendable metadata catalogue. Figure 3-1 shows an example of the metadata for one of GA's surveys. There are a number of S-57 type attributes, such as DRVAL1 and DRVAL2 to define the depth range value for the survey. GA have also added several of their own metadata fields, such as: The frequency and model of the sonar utilized for the survey: sonfrq and sonmod. The start and end port for the survey: porend and porsta The name of the vessel used for the survey: vesnam Fields highlighted in green in Figure 3-1 are mandatory attributes and must be populated for the survey to be loaded into the database. The management of this metadata ensures that the
data is easily accessible to all internal and external stakeholders through the use of structured database queries (e.g. find all surveys conducted in 2012 using a 12 khz sonar). Figure 3-1: Geoscience Australia customized metadata catalogue The use of batch scripts allow GA to batch load surveys into the database and automatically populate the attributes of each grid, saving time and eliminating the need to use the GUI. To assist with the implementation of the database solution, a senior technical consultant from CARIS HQ visited GA in a consultative capacity. Tasks undertaken included scheduling regular backups for the database, assistance in the construction of zone definition files (zdf), writing a script to automatically update zdf's and populating the database using the aforementioned batch scripts. 3.2 SURFACE OPERATIONS Two of the primary uses of BASE Manager by GA are to create and manage bathymetric grids from soundings and to compile and validate bathymetric data from multiple formats and sources. A number of surface functions are incorporated into the application to accomplish these tasks. It is often necessary to combine different bathymetric datasets into a seamless coverage, which can present many challenges as the data can be at different resolutions, with different coverages and varying levels of uncertainty. A number of deconfliction rules can be defined to determine which value is used in areas of overlap. Some commonly used rules include recent data (i.e. data that has the most recent collection date is favoured), least uncertainty (i.e. data that has the lowest level of uncertainty is favoured) and shoalest depth. Through the use of the deconfliction dialogue (see Figure 3-2), the user is given the power to define as many, or as few rules as needed for their requirements.
Figure 3-2: The Deconfliction Rule Wizard Figure 3-3 shows a surface that has been created from combining two surveys around Fraser Island. The first survey was collected in 2003 with a Reson Seabat 8101 (240 khz shallow water system) and provided to GA as an XYZ format and subsequently loaded into the database. The second survey was undertaken by GA in 2005 using an EM300 and loaded into the database as a CSAR file directly from HIPS. Figure 3-3: Combined surveys around Fraser Island The surface differencing tool has also been used to perform a comparison between the depth values of the two surveys (see Figure 3-4).
Figure 3-4: Difference surface of Fraser Island Surveys Both of these operations can be performed either locally on a desktop PC, or directly on the database to utilize the processing power of the server. 3.3 APPLICATION OF TIDES The BASE Manager client application has also been used to assist with the application of tides to surveys. Previously, a single tide station solution would be used for an entire survey, which could be problematic for surveys spanning over large or very long areas. BASE Manager was used to create and digitize tide zones (see Figure 3-5). This allowed GA to calculate load weighted estimates for tide in their surveys, providing much more accurate results (see Figure 3-5). As GA only use tide stations for surveys on or near the continental shelf, this could be taken into account in the zdf by defining an outer polygon that employs a zero tide rather than a weighted average from tide stations. These zdf files can also be used when processing raw survey data in the CARIS HIPS and SIPS application.
Figure 3-5: Digitized Tide Zones using BASE Manager
Figure 3-6: ZDF comparison using Tide Zones created in BASE Manager 3.4 CARIS SPATIAL ARCHIVE TECHNOLOGY To efficiently manage the increasing file sizes of modern day bathymetric data, CARIS redeveloped its data structures for gridded and point data using the CARIS Spatial Archive (CSAR) framework. The CSAR framework was designed to efficiently read, write and process the large volumes of data produced by modern survey sensors and methods. The CSAR framework works by partitioning data into pieces, called "chunks". Each chunk is given a unique key that can be used to retrieve it from a storage device such as a file or database (Masry, Collins et al., 2008). The division of data into chunks allows the efficient storage and visualization of high volumes of multi-dimensional data. Both gridded and point CSAR data structures are stored at multiple levels of resolution to facilitate rapid access. With the multiresolution structure, the data in a dataset is always available, although it is not always displayed. For example, if viewing a surface that is zoomed out, the surface displayed will be a coarse representation of the surface, appearing smooth and generalized. This coarse display will always include the shoalest depths. As you zoom in, the appearance of the surface will change as higher resolution data is brought into view. At any point, if you were to make a selection of points on the surface, you would get the same results whether zoomed in or out.
Proceedings of the Shallow Survey 2012 Conference CSAR data structures are created with a coverage polygon representing the boundaries of the data. This polygon is accessible through the bounding polygon layer available when a CSAR dataset is opened. The polygon includes the outer boundary of all data. The polygon can be edited, redigitized or removed. In 2005, Geoscience Australia and the National Oceans Office undertook a join project to produced a consistent, high-quality 9 arc second (approximately 250m at the equator) bathymetric grid for Australian waters (Whiteway, T., 2009). An updated version of the grid was released in 2009 incorporating a number of new datasets to ensure the most up to date data is available. The final grid is provided in a number of different formats, including a grid in ASCII XYZ format. This is made up of 16 tiles, with each tile containing 4 parts. In total, the grid contains 665,547,201 data points. As a test, the grid was imported into Bathy DataBASE as a point cloud to see how the CSAR framework would cope with the immense file size. The resulting CSAR file is approximately 25.5 Gb in size, and can be seen in 2D plan view (see Figure 3-7), as well as in 3D view (see Figure 3-8). Point clouds can also be interrogated in Subset Editor, allowing the user to reject, re-accept and designate soundings where required (see Figure 3-9). Figure 3-7: 2D Plan view of the Australian Bathymetry and Topography Grid 2009
Figure 3-8: 3D view of the Australian Bathymetry and Topography Grid 2009 Figure 3-9: Australian Bathymetry and Topography Grid 2009 in Subset Editor
4 CONCLUSION By migrating to a database solution, Geoscience Australia has greatly improved its ability to manage its bathymetric data. Through the use of an extensible metadata catalogue, the data has been made much more discoverable and easy to locate. Note: Geoscience Australia does not promote or endorse any specific software developer or hardware manufacturer such as CARIS or Kongsberg. The material in this presentation has been put forward as a case study. REFERENCES Masry, M., Collins, C., 2008, Scaling Bathymetry: Data handling for large volumes Whiteway, T., 2009, Australian Bathymetry and Topography Grid, June 2009, http://www.ga.gov.au/meta/anzcw0703013116.html