The NHD was designed and substantially built by three Federal partners: USGS, EPA and the USDA Forest Service. It was designed to be a reliable source of data that would grow both through system-wide revisions and the contributions of its users. While the Federal agencies generally represent a national perspective, thousands of other users have used the NHD to record and analyze waterways in their local areas. The data they have developed as users are then fed back to develop and refine the NHD as a whole.
Because the data will come increasingly from local sources, there will be a greater need for management and oversight at an intermediary level. Those intermediaries are referred to as "data stewards." The NHD will be maintained through data stewardship, a program in which the user community assumes more responsibility for the success of the NHD. Stewardship will be distributed across the Nation, typically on a State-by-State basis, and the principal stewards will coordinate and assume responsibility for the NHD at this level. Other organizations, which could have specific local or topical interests, may then assume further stewardship under the auspices of the principal stewards. This distributes stewardship down to the organizations that have the best available information on the hydrography in their area. These users can perform the actual enhancements with oversight and support from the principal stewards. These edits will become transactions provided to the NHD database that the USGS will maintain on behalf of the entire user community.
The NHD has been specifically designed to accommodate these transactions. Obviously, some changes can have significant impacts. Stream classification, for example, can affect natural resource management, flood hazard risk, development, storm water management, water rights, etc. It is essential that any changes made to the NHD are valid. The NHD also will depend on users not just to add and improve data, but to help identify policies and procedures that assure reliability and validity.
The USGS is already working with some agencies and groups that are interested in becoming NHD data stewards. Common needs and interests offer opportunities for partnerships to collect, maintain, access, and use basic spatial data among federal agencies and with other public organizations, notably State agencies or regional organizations. The most direct benefit of shared maintenance is the ability to have access to up-to-date information of changes on the landscape and to receive spatial data that faithfully represents those changes. The best sources for information about changes are those closest to the change, such as State and local governments.
The key to implementing stewardship is having an editing tool that can deal with the unique characteristics of NHD data. It will allow stewards to upgrade the NHD while preserving the integrity of the model and the geometric network. An editing tool specific to the NHD, known as NHDGeoEdit, was developed by ESRI for the Forest Service. USGS has assumed responsibility for maintaining and improving the tool. The tool will be available to the data stewards, but it does require some training to use. The USGS will train the principal stewards and they, in turn, will train their cadre of supporting organizations.
The Role of the Data Steward
The data steward's responsibilities include:
- Guarantee that updated NHD data pass all validation tests.
- Assure that the core content (features, attributes and relationships identified in the NHD standards) is included.
- Keep the data current.
- Consider any change submitted and decide authoritatively if it will be accepted or not. (Some changes can have significant impacts, and it is essential that any changes made to the NHD are valid.)
- Report the decision publicly.
- Respond to proposed changes within some agreed upon reasonable time.
- Provide publicly available information on status of data development and updating.
- Maintain awareness of activities by other agencies and groups.
Many people will be using the NHD for a variety of applications, some common and well understood and some unexpected. The user community must be confident that the updates will improve the quality of the data, and the data steward must assure the quality for all the users.
The USGS will provide tools to help maintain the NHD. These tools ensure that the updates are being managed correctly so the updates and the metadata associated with the updates can be easily incorporated into the national holdings. The USGS will provide training in the use of the tools and other NHD concepts. The USGS will provide support for applications such as reach indexing and basic hydrologic modeling.
The Importance of Change Management to the NHD
The National Hydrography Dataset is distinguished by several innovations that make it a cutting-edge GIS model. One of the most valuable is the ability to handle changes. Historically, edits--and re-edits--of GIS databases have put an unexpectedly high burden on resources. One of the biggest components of that burden was the cost incurred by failing to properly manage what had been edited. The NHD was designed to include data management mechanisms to help users and editors efficiently track the history of data edits. Since the NHD was intended to continually evolve, change management was of paramount concern.
Change management occurs at two levels: one is behind the scenes using ArcSDE data versioning, and the other is an explicit view of this process presented to the user. Basically, the NHD is designed to convert the behind-the-scenes record keeping to something the user can work with. Editing can take place as batch processing on the database to incorporate model upgrades, or it can be user-performed editing on the user's GIS. At either level, it is essential that the editing is managed to preserve and enhance data integrity.
If a user edits a stream, for example, five things must happen:
- the stream gets a new FDate,
- NHDMetadata gets a new record which will contain the metadata the user supplies, and the NHDFeatureToMetadata link gets a new record linking the feature to the new NHDMetadata record,
- if a new ReachCode is allocated to the stream or the stream is added to an existing ReachCode, a new cross-reference entry is made to NHDReachCrossReference, and
- if the stream edit resulted in a spatial change to an existing reach, the ReachSMDate in the NHDReachCode_ComID table is updated, and
- once the change is loaded into the NHD SDE production database, the database version number is changed (the version number is stored in the NHDProcessingParameter table in data that is distributed from the central database).
- The NHDGeoEdit tool will maintain the integrity of the NHD by managing the change process more or less automatically.
Overview of the NHDGeoEditTool
In order to make changes to the NHD manageable by a diverse group of data stewards and local editors, a tool was needed to automatically track the changes and populate all the necessary tables. ESRI developed the initial tool with funding from the USDA Forest Service. USGS has taken over responsibility for further development and improvement. USGS provides initial training on using the tools and will do follow-up training as necessary. Support is also available during the editing process.
The NHDGeoEditTool works with a personal geodatabase. It is managed with Task Assistant. The Task Assistant manages the tasks by NHD feature class and includes all the necessary editing functions for each feature class. Behind the scenes, the tool manages the NHD feature rules, and these rules must be applied for most edits. The tools also populate all the necessary tables. Some processes are performed outside of the tools, using standard ArcGIS edit tools. Good data management is required to manage the various files that are generated during the process, and a series of steps are needed before editing can begin. It is generally best to create all the necessary folders before editing begins. There is no one way to do this, but the USGS training provides one method that has worked for the USGS. The data to be edited must also be projected, because native NHD is in geographic coordinates. USGS supplies several templates, but any projection will work, and data stewards can determine which projection is best for them.
The NHDGeoEditTool must be loaded. Before any editing can begin, the metadata must be completed. A window pops up with all the necessary fields, and users can cut and paste information into these fields. In addition, the NHDProcessingParameters table must be updated with the name of the processing organization. This field is used by USGS to verify the updates are coming from the official data steward. Once the edits are complete, the data is unprojected back to geographic coordinates and an XML file of all the edits is generated. Only the XML file is actually required for the USGS SDE production database. These XML edits can also be loaded into the working copy of the data as a quality control check by the data steward. If everything looks acceptable, the XML file is transmitted to USGS and the edits are loaded. Generally, USGS loads the transactions and communicates the status of the edits to the data steward. Timing of the processing depends generally on other data in the queue. If there are no problems with the XML file, updates generally are available within two weeks.
Edit Transmission Techniques
When completed, the data steward informs the NHD POC and submits updates to a USGS NHD ftp site. USGS requests the final "geo_load.mdb", xml and rcl files. The status of the updates is tracked by the NHD POC. Stewards are notified if the load or validate fail and they receive the debug file. The steward makes required fixes and then resubmits the data for load.
The status layer on the NHD website was used to track the completion of the high-resolution. As the transition to maintenance is complete, the data stewards and USGS contact information is maintained and the status layer is used to track active updates and revisions.